API Reference

This page contains the complete API documentation for LLM Batch Helper.

Core Functions

llm_batch_helper.process_prompts_batch(prompts=None, input_dir=None, config=None, provider='openai', desc='Processing prompts', cache_dir=None, force=False)[source]

Process a batch of prompts through the LLM (synchronous version).

This is the main user-facing function that works in both regular Python scripts and Jupyter notebooks without requiring async/await syntax.

Parameters:

prompts (List[str | Tuple[str, str] | Dict[str, Any]] | None) – Optional list of prompts in any supported format (string, tuple, or dict)
input_dir (str | None) – Optional path to directory containing prompt files
config (LLMConfig) – LLM configuration
provider (str) – LLM provider to use (“openai”, “together”, “openrouter”, or “gemini”)
desc (str) – Description for progress bar
cache_dir (str | None) – Optional directory for caching responses
force (bool) – If True, force regeneration even if cached response exists

Returns:

Dict mapping prompt IDs to their responses, ordered by input sequence

Return type:

Dict[str, Dict[str, str | Dict]]

Note

Either prompts or input_dir must be provided, but not both. Results are returned in the same order as the input prompts.

Example

>>> from llm_batch_helper import LLMConfig, process_prompts_batch
>>> config = LLMConfig(model_name="gpt-4o-mini")
>>> results = process_prompts_batch(
...     prompts=["What is 2+2?", "What is the capital of France?"],
...     config=config,
...     provider="openai"
... )
>>> # Results will be in the same order as input prompts

Configuration

class llm_batch_helper.LLMConfig(model_name, temperature=1.0, max_tokens=None, system_instruction=None, max_retries=5, max_concurrent_requests=30, verification_callback=None, verification_callback_args=None, max_completion_tokens=None, **kwargs)[source]

Bases: object

Parameters:

model_name (str)
temperature (float)
max_tokens (int | None)
system_instruction (str | None)
max_retries (int)
max_concurrent_requests (int)
verification_callback (Callable[[...], bool] | None)
verification_callback_args (Dict | None)
max_completion_tokens (int | None)

Caching

class llm_batch_helper.LLMCache(cache_dir='llm_cache')[source]

Bases: object

Parameters:: cache_dir (str)

get_cached_response(prompt_id)[source]

Retrieve cached response if it exists.

Parameters:: prompt_id (str)
Return type:: Dict[str, Any] | None

save_response(prompt_id, prompt, response)[source]

Save response to cache.

Parameters:

prompt_id (str)
prompt (str)
response (Dict[str, Any])

Return type:

None

clear_cache()[source]

Clear all cached responses.

Return type:: None

Input Handlers

llm_batch_helper.get_prompts(input_source)[source]

Get prompts from either a directory or a list.

Parameters:: input_source (str | List[str | Tuple[str, str] | Dict[str, Any]]) – Either: - str: Path to directory containing prompt files - List: List of prompts in various formats (string, tuple, or dict)
Returns:: List of (prompt_id, prompt_text) tuples
Return type:: List[Tuple[str, str]]

llm_batch_helper.read_prompt_files(input_dir)[source]

Read all text files from input directory and return as (filename, content) pairs.

Parameters:: input_dir (str) – Path to directory containing prompt files
Returns:: List of (prompt_id, prompt_text) tuples where prompt_id is the filename without extension
Return type:: List[Tuple[str, str]]

llm_batch_helper.read_prompt_list(input_source)[source]

Read prompts from a list of various formats.

Parameters:: input_source (List[str | Tuple[str, str] | Dict[str, Any]]) – List of prompts in any of these formats: - str: The prompt text (will use hash as ID) - tuple: (prompt_id, prompt_text) - dict: {“id”: prompt_id, “text”: prompt_text}
Returns:: List of (prompt_id, prompt_text) tuples
Return type:: List[Tuple[str, str]]

Provider Functions

Internal provider functions (for advanced users):

llm_batch_helper.providers.log_retry_attempt(retry_state)[source]: Custom logging function for retry attempts.

async llm_batch_helper.providers.get_llm_response_with_internal_retry(prompt_id, prompt, config, provider, cache=None, force=False)[source]

Parameters:

prompt_id (str)
prompt (str)
config (LLMConfig)
provider (str)
cache (LLMCache | None)
force (bool)

Return type:

Dict[str, str | Dict]

async llm_batch_helper.providers.process_prompts_batch_async(prompts=None, input_dir=None, config=None, provider='openai', desc='Processing prompts', cache_dir=None, force=False)[source]

Process a batch of prompts through the LLM.

Parameters:

prompts (List[str | Tuple[str, str] | Dict[str, Any]] | None) – Optional list of prompts in any supported format (string, tuple, or dict)
input_dir (str | None) – Optional path to directory containing prompt files
config (LLMConfig) – LLM configuration
provider (str) – LLM provider to use (“openai”, “together”, “openrouter”, or “gemini”)
desc (str) – Description for progress bar
cache_dir (str | None) – Optional directory for caching responses
force (bool) – If True, force regeneration even if cached response exists

Returns:

Dict mapping prompt IDs to their responses, ordered by input sequence

Return type:

Dict[str, Dict[str, str | Dict]]

Note

Either prompts or input_dir must be provided, but not both. Results are returned in the same order as the input prompts.

llm_batch_helper.providers.process_prompts_batch(prompts=None, input_dir=None, config=None, provider='openai', desc='Processing prompts', cache_dir=None, force=False)[source]

Process a batch of prompts through the LLM (synchronous version).

This is the main user-facing function that works in both regular Python scripts and Jupyter notebooks without requiring async/await syntax.

Parameters:

prompts (List[str | Tuple[str, str] | Dict[str, Any]] | None) – Optional list of prompts in any supported format (string, tuple, or dict)
input_dir (str | None) – Optional path to directory containing prompt files
config (LLMConfig) – LLM configuration
provider (str) – LLM provider to use (“openai”, “together”, “openrouter”, or “gemini”)
desc (str) – Description for progress bar
cache_dir (str | None) – Optional directory for caching responses
force (bool) – If True, force regeneration even if cached response exists

Returns:

Dict mapping prompt IDs to their responses, ordered by input sequence

Return type:

Dict[str, Dict[str, str | Dict]]

Note

Either prompts or input_dir must be provided, but not both. Results are returned in the same order as the input prompts.

Example

>>> from llm_batch_helper import LLMConfig, process_prompts_batch
>>> config = LLMConfig(model_name="gpt-4o-mini")
>>> results = process_prompts_batch(
...     prompts=["What is 2+2?", "What is the capital of France?"],
...     config=config,
...     provider="openai"
... )
>>> # Results will be in the same order as input prompts

Exceptions

exception llm_batch_helper.exceptions.VerificationFailedError(message, prompt_id, llm_response_data=None)[source]

Bases: Exception

Custom exception for when verification callback fails.

exception llm_batch_helper.exceptions.InvalidPromptFormatError(message, invalid_item=None)[source]

Bases: Exception

Custom exception for invalid prompt format.