gen_ai_hub.batch_service package

Response returned by PATCH /llm-batch-service/v1/batches/{batch_id}/cancel.

Confirms that the cancellation request has been accepted. The job will transition to CANCELLING and eventually CANCELLED.

Parameters:

id (str, optional) -- Unique identifier (UUID) of the batch job.
created_at (str, optional) -- ISO 8601 timestamp of when the job was originally created.
message (str, optional) -- Human-readable confirmation that cancellation was scheduled.

created_at: str | None

id: str | None

message: str | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class BatchCreateRequest

Bases: ABCBaseModel

Request body sent to POST /llm-batch-service/v1/batches.

Describes a new batch processing job: where to read input from, where to write output, and which model to use.

Parameters:

type (Literal["llm-native"]) -- Batch processing type. Currently only "llm-native" is supported.
input (BatchInput) -- Input file configuration.
output (BatchOutput) -- Output directory configuration.
spec (BatchSpec) -- LLM provider and model specification.

input: BatchInput

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

output: BatchOutput

spec: BatchSpec

type: Literal['llm-native']

class BatchCreateResponse

Response returned by POST /llm-batch-service/v1/batches.

Confirms that the batch job has been accepted and provides the assigned identifier and initial status.

Parameters:

id (str) -- Unique identifier (UUID) of the created batch job.
created_at (str, optional) -- ISO 8601 timestamp of when the job was created.
status (str, optional) -- Initial status of the job, typically "PENDING".
message (str, optional) -- Human-readable confirmation message from the service.

created_at: str | None

id: str

message: str | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

status: str | None

class BatchDeleteResponse

Response returned by DELETE /llm-batch-service/v1/batches/{batch_id}.

Confirms that the batch job record has been deleted. Only jobs in a terminal state (COMPLETED, FAILED, or CANCELLED) can be deleted.

Parameters:

id (str, optional) -- Unique identifier (UUID) of the deleted batch job.
created_at (str, optional) -- ISO 8601 timestamp of when the job was originally created.
message (str, optional) -- Human-readable confirmation of the deletion.

created_at: str | None

id: str | None

message: str | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class BatchDetailResponse

Response returned by GET /llm-batch-service/v1/batches/{batch_id}.

Provides the full configuration and current status of a specific batch job.

Parameters:

id (str, optional) -- Unique identifier (UUID) of the batch job.
type (str, optional) -- Batch processing type (e.g. "llm-native").
provider (str, optional) -- LLM provider name (e.g. "azure-openai").
created_at (str, optional) -- ISO 8601 timestamp of when the job was created.
input (BatchInputDetail, optional) -- Input file configuration.
output (BatchOutputDetail, optional) -- Output directory configuration.
spec (dict, optional) -- Raw job specification dict as stored by the service.
status (BatchStatusDetail, optional) -- Current status details.

created_at: str | None

id: str | None

input: BatchInputDetail | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

output: BatchOutputDetail | None

provider: str | None

spec: dict | None

status: BatchStatusDetail | None

type: str | None

class BatchInput

Bases: ABCBaseModel

Input configuration for a batch job.

Points to the .jsonl file in an object store that contains the individual LLM requests to be processed.

Parameters:: uri (str) -- Fully qualified object-store URI of the input file. Must point to a .jsonl file (e.g. ai://my-store/input/requests.jsonl).

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

uri: str

class BatchInputDetail

Input configuration as returned in a batch detail response.

Parameters:: uri (str, optional) -- Object-store URI of the input .jsonl file.

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

uri: str | None

class BatchListResponse

Response returned by GET /llm-batch-service/v1/batches.

Contains a count and a list of batch job summaries for the current resource group.

Parameters:

count (int, optional) -- Total number of batch jobs.
resources (list[BatchSummary], optional) -- List of batch job summaries.

count: int | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

resources: list[BatchSummary] | None

class BatchOutput

Bases: ABCBaseModel

Output configuration for a batch job.

Points to the directory in an object store where results will be written once the job completes.

Parameters:: uri (str) -- Fully qualified object-store URI of the output directory (e.g. ai://my-store/output/).

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

uri: str

class BatchOutputDetail

Output configuration as returned in a batch detail response.

Parameters:: uri (str, optional) -- Object-store URI of the output directory.

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

uri: str | None

class BatchService

Bases: object

Client for the LLM Batch Service API.

Supports synchronous and asynchronous variants of all five operations: create, list, get, cancel, and delete batch jobs.

The AI-Resource-Group header is injected automatically from proxy_client.request_header on every request.

Parameters:

api_url (str, Optional) -- Base URL of the SAP AI Core API (e.g. https://api.ai.prod.eu-central-1.aws.ml.hana.ondemand.com/v2). Defaults to the URL resolved from proxy_client.
proxy_client (GenAIHubProxyClient) -- A GenAIHubProxyClient instance. Defaults to the result of get_proxy_client(proxy_version="gen-ai-hub").
resource_group (str, Optional) -- Value for the AI-Resource-Group header. Falls back to the resource group on proxy_client when omitted.
timeout (Union[int, float, httpx.Timeout], Optional) -- Default HTTP request timeout passed to httpx.

__init__(api_url=None, proxy_client=None, resource_group=None, timeout=None)

Parameters:

api_url (str | None)
proxy_client (GenAIHubProxyClient | None)
resource_group (str | None)
timeout (int | float | Timeout | None)

async acancel(batch_id, timeout=None)

Async variant of cancel().

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchCancelResponse confirming the cancellation request.

Return type:

async aclose_http_connection()

Close the underlying asynchronous httpx client.

Return type:: None

async acreate(*, type='llm-native', input_uri, output_uri, provider, model, timeout=None)

Async variant of create().

Parameters:

type (str) -- Batch processing type (only "llm-native" is supported).
input_uri (str) -- URI of the input .jsonl file in the object store.
output_uri (str) -- URI of the output directory in the object store.
provider (str) -- LLM provider name (e.g. "azure-openai").
model (str) -- Model name (e.g. "gpt-4.1-mini").
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchCreateResponse with the job ID and initial status.

Return type:

async adelete(batch_id, timeout=None)

Async variant of delete().

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchDeleteResponse confirming the deletion.

Return type:

async aget(batch_id, timeout=None)

Async variant of get().

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchDetailResponse with full job details.

Return type:

async aget_status(batch_id, timeout=None)

Async variant of get_status().

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchStatusResponse with current and target status.

Return type:

async alist(timeout=None)

Async variant of list().

Parameters:: timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.
Returns:: BatchListResponse containing the batch summaries.
Return type:: BatchListResponse

cancel(batch_id, timeout=None)

Schedule a batch job for cancellation.

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchCancelResponse confirming the cancellation request.

Return type:

close_http_connection()

Close the underlying synchronous httpx client.

Return type:: None

create(*, type, input_uri, output_uri, provider, model, timeout=None)

Create a new batch processing job.

Parameters:

type (str) -- Batch processing type (only "llm-native" is supported).
input_uri (str) -- URI of the input .jsonl file in the object store.
output_uri (str) -- URI of the output directory in the object store.
provider (str) -- LLM provider name (e.g. "azure-openai").
model (str) -- Model name (e.g. "gpt-4.1-mini").
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchCreateResponse with the job ID and initial status.

Return type:

delete(batch_id, timeout=None)

Delete a batch job (only allowed for terminal states: COMPLETED, FAILED, CANCELLED).

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchDeleteResponse confirming the deletion.

Return type:

get(batch_id, timeout=None)

Retrieve details of a specific batch job.

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchDetailResponse with full job details.

Return type:

get_status(batch_id, timeout=None)

Retrieve the current status of a batch job.

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchStatusResponse with current and target status.

Return type:

list(timeout=None)

List all batch jobs for the current resource group.

Parameters:: timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.
Returns:: BatchListResponse containing the batch summaries.
Return type:: BatchListResponse

class BatchSpec

Bases: ABCBaseModel

Specification of the LLM to use for a batch job.

Parameters:

provider (str) -- LLM provider name as registered in SAP AI Core (e.g. "azure-openai").
model (str) -- Model name to use for inference (e.g. "gpt-4.1-mini").

model: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

provider: str

class BatchStatus

Bases: str, Enum

Enumeration of possible lifecycle states for a batch job.

Variables:

PENDING -- Job has been accepted and is waiting to be scheduled.
RUNNING -- Job is actively being processed.
COMPLETED -- Job finished successfully.
FAILED -- Job terminated with an error.
CANCELLED -- Job was cancelled by the user.
CANCELLING -- Cancellation has been requested and is in progress.

__new__(value)

CANCELLED = 'CANCELLED'

CANCELLING = 'CANCELLING'

COMPLETED = 'COMPLETED'

FAILED = 'FAILED'

PENDING = 'PENDING'

RUNNING = 'RUNNING'

class BatchStatusDetail

Status block embedded inside BatchDetailResponse.

Parameters:

current_status (str, optional) -- The job's current lifecycle status.
target_status (str, optional) -- The terminal status the job is expected to reach.
updated_at (str, optional) -- ISO 8601 timestamp of the last status change.
message (str, optional) -- Optional human-readable description of the current status.

current_status: str | None

message: str | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

target_status: str | None

updated_at: str | None

class BatchStatusResponse

Response returned by GET /llm-batch-service/v1/batches/{batch_id}/status.

Parameters:

current_status (str, optional) -- The job's current lifecycle status.
target_status (str, optional) -- The terminal status the job is expected to reach.
updated_at (str, optional) -- ISO 8601 timestamp of the last status change.
message (str, optional) -- Optional human-readable description of the current status.

current_status: str | None

message: str | None

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

target_status: str | None

updated_at: str | None

class BatchSummary

Summary entry for a single batch job as returned in a list response.

Parameters:

id (str) -- Unique identifier (UUID) of the batch job.
type (str, optional) -- Batch processing type (e.g. "llm-native").
provider (str, optional) -- LLM provider name (e.g. "azure-openai").
created_at (str, optional) -- ISO 8601 timestamp of when the job was created.
status (str, optional) -- Current status of the job.

created_at: str | None

id: str

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

provider: str | None

status: str | None

type: str | None

class ErrorResponse

Error response body returned by the batch service on 4xx/5xx responses.

Parameters:

request_id (str) -- Unique request identifier, useful for tracing the error in service logs.
message (str) -- Human-readable description of the error.

message: str

model_config: ClassVar[ConfigDict] = {'extra': 'allow', 'frozen': False}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str

Subpackages

gen_ai_hub.batch_service.models package

Submodules

gen_ai_hub.batch_service.exceptions module

Exceptions for the batch service module.

exception BatchServiceError

Bases: Exception

Raised when the batch service returns an error response.

Captures the request_id from the error payload for tracing.

__init__(request_id, message, status_code, headers)

Parameters:

request_id (str)
message (str)
status_code (int)
headers (Headers)

gen_ai_hub.batch_service.service module

Client for the LLM Batch Service API.

Provides synchronous and asynchronous methods to create, list, inspect, cancel, and delete batch processing jobs via SAP AI Core.

class BatchService

Bases: object

Client for the LLM Batch Service API.

Supports synchronous and asynchronous variants of all five operations: create, list, get, cancel, and delete batch jobs.

The AI-Resource-Group header is injected automatically from proxy_client.request_header on every request.

Parameters:

api_url (str, Optional) -- Base URL of the SAP AI Core API (e.g. https://api.ai.prod.eu-central-1.aws.ml.hana.ondemand.com/v2). Defaults to the URL resolved from proxy_client.
proxy_client (GenAIHubProxyClient) -- A GenAIHubProxyClient instance. Defaults to the result of get_proxy_client(proxy_version="gen-ai-hub").
resource_group (str, Optional) -- Value for the AI-Resource-Group header. Falls back to the resource group on proxy_client when omitted.
timeout (Union[int, float, httpx.Timeout], Optional) -- Default HTTP request timeout passed to httpx.

__init__(api_url=None, proxy_client=None, resource_group=None, timeout=None)

Parameters:

api_url (str | None)
proxy_client (GenAIHubProxyClient | None)
resource_group (str | None)
timeout (int | float | Timeout | None)

async acancel(batch_id, timeout=None)

Async variant of cancel().

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchCancelResponse confirming the cancellation request.

Return type:

async aclose_http_connection()

Close the underlying asynchronous httpx client.

Return type:: None

async acreate(*, type='llm-native', input_uri, output_uri, provider, model, timeout=None)

Async variant of create().

Parameters:

type (str) -- Batch processing type (only "llm-native" is supported).
input_uri (str) -- URI of the input .jsonl file in the object store.
output_uri (str) -- URI of the output directory in the object store.
provider (str) -- LLM provider name (e.g. "azure-openai").
model (str) -- Model name (e.g. "gpt-4.1-mini").
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchCreateResponse with the job ID and initial status.

Return type:

async adelete(batch_id, timeout=None)

Async variant of delete().

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchDeleteResponse confirming the deletion.

Return type:

async aget(batch_id, timeout=None)

Async variant of get().

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchDetailResponse with full job details.

Return type:

async aget_status(batch_id, timeout=None)

Async variant of get_status().

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchStatusResponse with current and target status.

Return type:

async alist(timeout=None)

Async variant of list().

Parameters:: timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.
Returns:: BatchListResponse containing the batch summaries.
Return type:: BatchListResponse

cancel(batch_id, timeout=None)

Schedule a batch job for cancellation.

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchCancelResponse confirming the cancellation request.

Return type:

close_http_connection()

Close the underlying synchronous httpx client.

Return type:: None

create(*, type, input_uri, output_uri, provider, model, timeout=None)

Create a new batch processing job.

Parameters:

type (str) -- Batch processing type (only "llm-native" is supported).
input_uri (str) -- URI of the input .jsonl file in the object store.
output_uri (str) -- URI of the output directory in the object store.
provider (str) -- LLM provider name (e.g. "azure-openai").
model (str) -- Model name (e.g. "gpt-4.1-mini").
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchCreateResponse with the job ID and initial status.

Return type:

delete(batch_id, timeout=None)

Delete a batch job (only allowed for terminal states: COMPLETED, FAILED, CANCELLED).

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchDeleteResponse confirming the deletion.

Return type:

get(batch_id, timeout=None)

Retrieve details of a specific batch job.

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchDetailResponse with full job details.

Return type:

get_status(batch_id, timeout=None)

Retrieve the current status of a batch job.

Parameters:

batch_id (str) -- UUID of the batch job.
timeout (Union[int, float, httpx.Timeout], Optional) -- Per-request timeout override.

Returns:

BatchStatusResponse with current and target status.

Return type: