API Reference

This page contains the complete API documentation for LLM Batch Helper.

Core Functions

llm_batch_helper.process_prompts_batch(prompts=None, messages=None, input_dir=None, config=None, provider='openai', desc='Processing prompts', cache_dir=None, force=False)

Process a batch of prompts or messages through the LLM (synchronous version).

This is the main user-facing function that works in both regular Python scripts and Jupyter notebooks without requiring async/await syntax.

Parameters:

• prompts (List[str | Tuple[str, str] | Dict[str, Any]] | None) – Optional list of prompts in any supported format (string, tuple, or dict)

• messages (List[Tuple[str, List[Dict[str, str]]] | Dict[str, Any]] | None) – Optional list of messages in format: [(message_id, [{‘role’: ‘user’, ‘content’: ‘…’}])]

• input_dir (str | None) – Optional path to directory containing prompt files

• config (LLMConfig) – LLM configuration

• provider (str) – LLM provider to use (“openai”, “together”, “openrouter”, or “gemini”)

• desc (str) – Description for progress bar

• cache_dir (str | None) – Optional directory for caching responses

• force (bool) – If True, force regeneration even if cached response exists

Returns:

Dict mapping prompt/message IDs to their responses, ordered by input sequence

Return type:

Dict[str, Dict[str, str | Dict]]

Note

Either prompts, messages, or input_dir must be provided, but not multiple. Results are returned in the same order as the input prompts/messages.

Example

>>> from llm_batch_helper import LLMConfig, process_prompts_batch
>>> config = LLMConfig(model_name="gpt-4o-mini")
>>> # Prompt mode
>>> results = process_prompts_batch(
...     prompts=["What is 2+2?", "What is the capital of France?"],
...     config=config,
...     provider="openai"
... )
>>> # Message mode
>>> results = process_prompts_batch(
...     messages=[("msg1", [{"role": "user", "content": "Hello!"}])],
...     config=config,
...     provider="openai"
... )

Configuration

class llm_batch_helper.LLMConfig(model_name, temperature=1.0, max_tokens=None, system_instruction=None, max_retries=5, max_concurrent_requests=30, verification_callback=None, verification_callback_args=None, max_completion_tokens=None, **kwargs)

Bases: object

Parameters:

• model_name (str)

• temperature (float)

• max_tokens (int | None)

• system_instruction (str | None)

• max_retries (int)

• max_concurrent_requests (int)

• verification_callback (Callable[[...], bool] | None)

• verification_callback_args (Dict | None)

• max_completion_tokens (int | None)

Caching

class llm_batch_helper.LLMCache(cache_dir='llm_cache')

Bases: object

Parameters:

cache_dir (str)

get_cached_response(prompt_id)

Retrieve cached response if it exists.

Parameters:

prompt_id (str)

Return type:

Dict[str, Any] | None

save_response(prompt_id, prompt, response)

Save response to cache.

Parameters:

• prompt_id (str)

• prompt (str)

• response (Dict[str, Any])

Return type:

None

clear_cache()

Clear all cached responses.

Return type:

None

Input Handlers

llm_batch_helper.get_prompts(input_source)

Get prompts from either a directory or a list.

Parameters:

input_source (str | List[str | Tuple[str, str] | Dict[str, Any]]) – Either: - str: Path to directory containing prompt files - List: List of prompts in various formats (string, tuple, or dict)

Returns:

List of (prompt_id, prompt_text) tuples

Return type:

List[Tuple[str, str]]

llm_batch_helper.read_prompt_files(input_dir)

Read all text files from input directory and return as (filename, content) pairs.

Parameters:

input_dir (str) – Path to directory containing prompt files

Returns:

List of (prompt_id, prompt_text) tuples where prompt_id is the filename without extension

Return type:

List[Tuple[str, str]]

llm_batch_helper.read_prompt_list(input_source)

Read prompts from a list of various formats.

Parameters:

input_source (List[str | Tuple[str, str] | Dict[str, Any]]) – List of prompts in any of these formats: - str: The prompt text (will use hash as ID) - tuple: (prompt_id, prompt_text) - dict: {“id”: prompt_id, “text”: prompt_text}

Returns:

List of (prompt_id, prompt_text) tuples

Return type:

List[Tuple[str, str]]

Provider Functions

Internal provider functions (for advanced users):

llm_batch_helper.providers.log_retry_attempt(retry_state)

Custom logging function for retry attempts.

async llm_batch_helper.providers.get_llm_response_with_internal_retry(prompt_id, prompt, config, provider, cache=None, force=False, messages=None)

Parameters:

• prompt_id (str)

• prompt (str)

• config (LLMConfig)

• provider (str)

• cache (LLMCache | None)

• force (bool)

• messages (List[Dict[str, str]] | None)

Return type:

Dict[str, str | Dict]

async llm_batch_helper.providers.process_prompts_batch_async(prompts=None, messages=None, input_dir=None, config=None, provider='openai', desc='Processing prompts', cache_dir=None, force=False)

Process a batch of prompts or messages through the LLM.

Parameters:

• prompts (List[str | Tuple[str, str] | Dict[str, Any]] | None) – Optional list of prompts in any supported format (string, tuple, or dict)

• messages (List[Tuple[str, List[Dict[str, str]]] | Dict[str, Any]] | None) – Optional list of messages in format: [(message_id, [{‘role’: ‘user’, ‘content’: ‘…’}])]

• input_dir (str | None) – Optional path to directory containing prompt files

• config (LLMConfig) – LLM configuration

• provider (str) – LLM provider to use (“openai”, “together”, “openrouter”, or “gemini”)

• desc (str) – Description for progress bar

• cache_dir (str | None) – Optional directory for caching responses

• force (bool) – If True, force regeneration even if cached response exists

Returns:

Dict mapping prompt/message IDs to their responses, ordered by input sequence

Return type:

Dict[str, Dict[str, str | Dict]]

Note

Either prompts, messages, or input_dir must be provided, but not multiple. Results are returned in the same order as the input prompts/messages.

llm_batch_helper.providers.process_prompts_batch(prompts=None, messages=None, input_dir=None, config=None, provider='openai', desc='Processing prompts', cache_dir=None, force=False)

Process a batch of prompts or messages through the LLM (synchronous version).

This is the main user-facing function that works in both regular Python scripts and Jupyter notebooks without requiring async/await syntax.

Parameters:

• prompts (List[str | Tuple[str, str] | Dict[str, Any]] | None) – Optional list of prompts in any supported format (string, tuple, or dict)

• messages (List[Tuple[str, List[Dict[str, str]]] | Dict[str, Any]] | None) – Optional list of messages in format: [(message_id, [{‘role’: ‘user’, ‘content’: ‘…’}])]

• input_dir (str | None) – Optional path to directory containing prompt files

• config (LLMConfig) – LLM configuration

• provider (str) – LLM provider to use (“openai”, “together”, “openrouter”, or “gemini”)

• desc (str) – Description for progress bar

• cache_dir (str | None) – Optional directory for caching responses

• force (bool) – If True, force regeneration even if cached response exists

Returns:

Dict mapping prompt/message IDs to their responses, ordered by input sequence

Return type:

Dict[str, Dict[str, str | Dict]]

Note

Either prompts, messages, or input_dir must be provided, but not multiple. Results are returned in the same order as the input prompts/messages.

Example

>>> from llm_batch_helper import LLMConfig, process_prompts_batch
>>> config = LLMConfig(model_name="gpt-4o-mini")
>>> # Prompt mode
>>> results = process_prompts_batch(
...     prompts=["What is 2+2?", "What is the capital of France?"],
...     config=config,
...     provider="openai"
... )
>>> # Message mode
>>> results = process_prompts_batch(
...     messages=[("msg1", [{"role": "user", "content": "Hello!"}])],
...     config=config,
...     provider="openai"
... )

Exceptions

exception llm_batch_helper.exceptions.VerificationFailedError(message, prompt_id, llm_response_data=None)

Bases: Exception

Custom exception for when verification callback fails.

exception llm_batch_helper.exceptions.InvalidPromptFormatError(message, invalid_item=None)

Bases: Exception

Custom exception for invalid prompt format.