gen_ai_hub.orchestration_v2.models package

Submodules

gen_ai_hub.orchestration_v2.models.azure_content_filter module

Azure Content Filter Model

class AzureContentFilter

Bases: ABCBaseModel

Specific filter configuration for Azure Content Safety.

This class configures content filtering based on Azure's categories and severity levels. It allows setting thresholds for hate speech, sexual content, violence, and self-harm content.

Args:

hate: Threshold for hate speech content. sexual: Threshold for sexual content. violence: Threshold for violent content. self_harm: Threshold for self-harm content. prompt_shield: A flag to use prompt shield

hate: AzureThreshold | Literal[0, 2, 4, 6] | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

self_harm: AzureThreshold | Literal[0, 2, 4, 6] | None

sexual: AzureThreshold | Literal[0, 2, 4, 6] | None

violence: AzureThreshold | Literal[0, 2, 4, 6] | None

class AzureContentSafetyInput

Bases: AzureContentFilter

Filter configuration for Azure Content Safety Input

Args:

hate: Threshold for hate speech content. sexual: Threshold for sexual content. violence: Threshold for violent content. self_harm: Threshold for self-harm content. prompt_shield: A flag to use prompt shield

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

prompt_shield: bool | None

class AzureContentSafetyOutput

Bases: AzureContentFilter

Filter configuration for Azure Content Safety Output

Args:

hate: Threshold for hate speech content. sexual: Threshold for sexual content. violence: Threshold for violent content. self_harm: Threshold for self-harm content. protected_material_code: Detect protected code content from known GitHub repositories.

The scan includes software libraries, source code, algorithms, and other proprietary programming content.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

protected_material_code: bool | None

class AzureThreshold

Bases: int, Enum

Enumerates the threshold levels for the Azure Content Safety service.

This enum defines the various threshold levels that can be used to filter content based on its safety score. Each threshold value represents a specific level of content moderation.

Values:

ALLOW_SAFE: Allows only Safe content. ALLOW_SAFE_LOW: Allows Safe and Low content. ALLOW_SAFE_LOW_MEDIUM: Allows Safe, Low, and Medium content. ALLOW_ALL: Allows all content (Safe, Low, Medium, and High).

__new__(value)

ALLOW_ALL = 6

ALLOW_SAFE = 0

ALLOW_SAFE_LOW = 2

ALLOW_SAFE_LOW_MEDIUM = 4

gen_ai_hub.orchestration_v2.models.base module

A base model class that extends Pydantic's BaseModel and ABC.

class ABCBaseModel

Bases: BaseModel, ABC

Abstract base model that extends Pydantic's BaseModel and ABC.

• extra="forbid" prevents unexpected fields from being accepted or sent, since the external API is strict about allowed parameters.

• exclude_none=True ensures optional fields that were not provided are omitted from the payload instead of being serialized as None.

• by_alias=True guarantees aliases are used during serialization, which is required because some API field names conflict with Pydantic's built-ins.

This enforces consistent and safe serialization behavior across all derived models.

model_dump(**kwargs)

Dumps the model to a dictionary with default settings.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

gen_ai_hub.orchestration_v2.models.config module

Orchestration Service configuration models.

class CompletionRequestConfigurationReferenceByIdConfigRef

Bases: ABCBaseModel

Represents a reference to an orchestration config identified by a unique ID.

Args:

id (str): The unique identifier for the configuration.

id: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class CompletionRequestConfigurationReferenceByNameScenarioVersionConfigRef

Bases: ABCBaseModel

Represents a reference to aan orchestration config identified by name, scenario, and version.

Args:

scenario(str): Scenario name

name(str): Name of config

version(str): Version of config

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str

scenario: str

version: str

class ModuleConfig

Bases: ABCBaseModel

Configuration for the Orchestration Service's content generation process.

Defines modules for a harmonized API that combines LLM-based content generation with additional processing functionalities.

The orchestration service allows for advanced content generation by processing inputs through a series of steps: template rendering, text generation via LLMs, and optional input/output transformations such as data masking or filtering.

Args:

prompt_templating: Template object for rendering input prompts and language model for text generation.

filtering: Module for filtering and validating input/output content.

masking: Module for anonymizing or pseudonymizing sensitive information.

grounding: Module for document grounding.

translation: Module for translating input and output content.

filtering: FilteringModuleConfig | None

grounding: GroundingModuleConfig | None

masking: MaskingModuleConfig | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

prompt_templating: PromptTemplatingModuleConfig

translation: TranslationModuleConfig | None

class OrchestrationConfig

Bases: ABCBaseModel

Configuration for the Orchestration Service's content generation process.

Args:

modules: Either a single ModuleConfig or a list of ModuleConfigs. When a list is provided, the orchestration service will try each configuration in order until one succeeds.

stream: Optional streaming configuration.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

modules: ModuleConfig | Annotated[List[ModuleConfig], FieldInfo(annotation=NoneType, required=True, metadata=[MinLen(min_length=1)])]

stream: GlobalStreamOptions | None

gen_ai_hub.orchestration_v2.models.content_filter module

Content filter models for various providers.

class AzureContentSafetyInputFilterConfig

Bases: ContentFilter

config: AzureContentSafetyInput | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ContentFilterProvider

class AzureContentSafetyOutputFilterConfig

Bases: ContentFilter

config: AzureContentSafetyOutput | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ContentFilterProvider

class ContentFilter

Bases: ABCBaseModel

Base class for content filtering configurations.

This class provides a generic structure for defining content filters from various providers. It allows for specifying the provider and associated configuration parameters.

Args:

type: The name of the content filter provider.

config: A dictionary containing the configuration parameters for the content filter.

config: AzureContentFilter | LlamaGuard38bFilter | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ContentFilterProvider

class ContentFilterProvider

Bases: str, Enum

Enumerates supported content filter providers.

This enum defines the available content filtering services that can be used for content moderation tasks. Each enum value represents a specific provider.

Values:

AZURE: Represents the Azure Content Safety service.

LLAMA_GUARD_3_8B: Represents the Llama Guard 3 based on Llama-3.1-8B pretrained model.

__new__(value)

AZURE = 'azure_content_safety'

LLAMA_GUARD_3_8B = 'llama_guard_3_8b'

class FilteringStreamOptions

Bases: ABCBaseModel

overlap: Number of characters that should be additionally sent to content filtering services from previous chunks as additional context.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

overlap: int | None

class LlamaGuard38bFilterConfig

Bases: ContentFilter

config: LlamaGuard38bFilter

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ContentFilterProvider

gen_ai_hub.orchestration_v2.models.content_filtering module

Module for managing and applying content filters in the orchestration system.

class FilteringModuleConfig

Bases: ABCBaseModel

Module for managing and applying content filters.

Args:

input: Module for filtering and validating input content before processing.

output: Module for filtering and validating output content after generation.

classmethod enforce_min_properties(values)

Ensure at least one of input or output filtering is provided.

input: InputFiltering | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

output: OutputFiltering | None

class InputFiltering

Bases: ABCBaseModel

Module for managing and applying input content filters.

Args:

filters: List of ContentFilter objects to be applied to input content.

filters: List[AzureContentSafetyInputFilterConfig | LlamaGuard38bFilterConfig | ContentFilter]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class OutputFiltering

Bases: ABCBaseModel

Module for managing and applying output content filters.

Args:

filters: List of ContentFilter objects to be applied to output content.

stream_options: Module-specific streaming options.

filters: List[AzureContentSafetyOutputFilterConfig | LlamaGuard38bFilterConfig | ContentFilter]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

stream_options: FilteringStreamOptions | None

gen_ai_hub.orchestration_v2.models.data_masking module

Data Masking Module Configuration Models

class DPICustomEntity

Bases: ABCBaseModel

regex: Regular expression to match the entity replacement_strategy: Replacement strategy to be used for the entity

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

regex: str

replacement_strategy: DPIMethodConstant

class DPIMethodConstant

Bases: ABCBaseModel

Replaces the entity with the specified value followed by an incrementing number

method: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

value: str

class DPIMethodFabricatedData

Bases: ABCBaseModel

Replaces the entity with a randomly generated value appropriate to its type.

method: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class DPIStandardEntity

Bases: ABCBaseModel

type: Standard entity type to be masked replacement_strategy: Replacement strategy to be used for the entity

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

replacement_strategy: DPIMethodConstant | DPIMethodFabricatedData | None

type_: ProfileEntity

class DataMaskingProviderName

Bases: str, Enum

Enumerates the available data masking providers.

This enum defines the supported providers for masking sensitive data in the LLM module.

Values: SAP_DATA_PRIVACY_INTEGRATION: Refers to the SAP Data Privacy Integration service, which offers anonymization and pseudonymization capabilities for sensitive data.

__new__(value)

SAP_DATA_PRIVACY_INTEGRATION = 'sap_data_privacy_integration'

class MaskGroundingInput

Bases: ABCBaseModel

Controls whether the input to the grounding module will be masked with the configuration supplied in the masking module

enabled: bool

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class MaskingMethod

Bases: str, Enum

Enumerates the supported masking methods.

This enum defines the two main methods for masking sensitive information: anonymization and pseudonymization. Anonymization irreversibly removes sensitive data, while pseudonymization allows the original data to be recovered.

Values:

ANONYMIZATION: Irreversibly replaces sensitive data with placeholders (e.g., MASKED_ENTITY).

PSEUDONYMIZATION: Replaces sensitive data with reversible placeholders (e.g., MASKED_ENTITY_ID).

__new__(value)

ANONYMIZATION = 'anonymization'

PSEUDONYMIZATION = 'pseudonymization'

class MaskingModuleConfig

Bases: ABCBaseModel

Configuration for the data masking module.

Args:

providers: list of masking service provider configurations masking_providers: list of masking provider configurations

IMPORTANT: use exactly one of the parameters to set the list of masking provider configurations. DEPRECATED: parameter 'masking_providers' will be removed Sept 15, 2026. Use 'providers' instead.

enforce_exactly_one_provider_list()

masking_providers: List[MaskingProviderConfig] | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

providers: List[MaskingProviderConfig] | None

class MaskingProviderConfig

Bases: ABCBaseModel

SAP Data Privacy Integration provider for data masking.

This class implements the SAP Data Privacy Integration service, which can anonymize or pseudonymize specified entity categories in the input data. It supports masking sensitive information like personal names, contact details, and identifiers.

Args:

method: The method of masking to apply (anonymization or pseudonymization).

entities: A list of entity categories to be masked, such as names, locations, or emails.

allowlist: A list of strings that should not be masked.

mask_grounding_input: A flag indicating whether to mask input to the grounding module.

allowlist: List[str] | None

entities: List[DPIStandardEntity | DPICustomEntity]

mask_grounding_input: MaskGroundingInput | None

method: MaskingMethod

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: DataMaskingProviderName

class ProfileEntity

Bases: str, Enum

Enumerates the entity categories that can be masked by the SAP Data Privacy Integration service.

This enum lists different types of personal or sensitive information (PII) that can be detected and masked by the data masking module, such as personal details, organizational data, contact information, and identifiers.

Values:

PERSON: Represents personal names.

ORG: Represents organizational names.

UNIVERSITY: Represents educational institutions.

LOCATION: Represents geographical locations.

EMAIL: Represents email addresses.

PHONE: Represents phone numbers.

ADDRESS: Represents physical addresses.

SAP_IDS_INTERNAL: Represents internal SAP identifiers.

SAP_IDS_PUBLIC: Represents public SAP identifiers.

URL: Represents URLs.

USERNAME_PASSWORD: Represents usernames and passwords.

NATIONAL_ID: Represents national identification numbers.

IBAN: Represents International Bank Account Numbers.

SSN: Represents Social Security Numbers.

CREDIT_CARD_NUMBER: Represents credit card numbers.

PASSPORT: Represents passport numbers.

DRIVING_LICENSE: Represents driving license numbers.

NATIONALITY: Represents nationality information.

RELIGIOUS_GROUP: Represents religious group affiliation.

POLITICAL_GROUP: Represents political group affiliation.

PRONOUNS_GENDER: Represents pronouns and gender identity.

GENDER: Represents gender information.

SEXUAL_ORIENTATION: Represents sexual orientation.

TRADE_UNION: Represents trade union membership.

SENSITIVE_DATA: Represents any other sensitive information.

__new__(value)

ADDRESS = 'profile-address'

CREDIT_CARD_NUMBER = 'profile-credit-card-number'

DRIVING_LICENSE = 'profile-driverlicense'

EMAIL = 'profile-email'

ETHNICITY = 'profile-ethnicity'

GENDER = 'profile-gender'

IBAN = 'profile-iban'

LOCATION = 'profile-location'

NATIONALITY = 'profile-nationality'

NATIONAL_ID = 'profile-nationalid'

ORG = 'profile-org'

PASSPORT = 'profile-passport'

PERSON = 'profile-person'

PHONE = 'profile-phone'

POLITICAL_GROUP = 'profile-political-group'

PRONOUNS_GENDER = 'profile-pronouns-gender'

RELIGIOUS_GROUP = 'profile-religious-group'

SAP_IDS_INTERNAL = 'profile-sapids-internal'

SAP_IDS_PUBLIC = 'profile-sapids-public'

SENSITIVE_DATA = 'profile-sensitive-data'

SEXUAL_ORIENTATION = 'profile-sexual-orientation'

SSN = 'profile-ssn'

TRADE_UNION = 'profile-trade-union'

UNIVERSITY = 'profile-university'

URL = 'profile-url'

USERNAME_PASSWORD = 'profile-username-password'

gen_ai_hub.orchestration_v2.models.document_grounding module

Module for defining document grounding configurations.

class DataRepositoryType

Bases: str, Enum

Enumerates data repository types.

__new__(value)

URL = 'help.sap.com'

VECTOR = 'vector'

class DocumentGroundingConfig

Bases: ABCBaseModel

defines the detailed configuration for the Grounding module.

Args:

filters: List of DocumentGroundingFilter objects.

placeholders: Placeholders to be used for grounding input questions and output.

metadata_params: Parameter name used for specifying metadata parameters.

filters: List[DocumentGroundingFilter] | None

metadata_params: list[str] | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

placeholders: DocumentGroundingPlaceholders

class DocumentGroundingFilter

Bases: ABCBaseModel

Module for configuring document grounding filters.

Args:

id: The unique identifier for the grounding filter.

search_config: GroundingSearchConfig object.

data_repository_type: Only include DataRepositories with the given type:

vector, help.sap.com.

data_repositories: list of data repositories to search.

Specify ['*'] to search across all DataRepositories or give a specific list of DataRepository ids.

data_repository_metadata: The metadata for the data repository.

Restrict DataRepositories considered during search to those annotated with the given metadata. Useful when combined with dataRepositories=['*']

document_metadata: DocumentMetadata object.

chunk_metadata: Restrict chunks considered during search to those with the given metadata.

chunk_metadata: List[KeyValueListPair] | None

data_repositories: List[str] | None

data_repository_metadata: List[KeyValueListPair] | None

data_repository_type: DataRepositoryType | Literal['vector', 'help.sap.com']

document_metadata: List[DocumentMetadataKeyValueListPairs] | None

id: str | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

search_config: GroundingSearchConfig | None

class DocumentGroundingPlaceholders

Bases: ABCBaseModel

input: The list of input parameters used for grounding input questions (minItems: 1). output: Parameter name used for grounding output.

input: List[str]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

output: str

class DocumentMetadataKeyValueListPairs

Bases: KeyValueListPair

Restrict documents considered during search to those annotated with the given metadata.

Args:

key: The key for the metadata.

value: The list of values for the metadata.

select_mode: Select mode for search filters.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

select_mode: List[Literal['ignoreIfKeyAbsent']] | None

class GroundingModuleConfig

Bases: ABCBaseModel

Module for managing and applying grounding aka RAG configurations.

Args:

type: The type of the grounding module.

config: Configuration dictionary for the grounding module.

config: DocumentGroundingConfig

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type: GroundingType

class GroundingSearchConfig

Bases: ABCBaseModel

Search configuration for the data repository.

Args:

max_chunk_count(int, minimum: 0, exclusiveMinimum: true): Maximum number of chunks to be returned. Cannot be used with 'maxDocumentCount'.

max_document_count(int, minimum: 0, exclusiveMinimum: true): [Only supports 'vector' dataRepositoryType] - Maximum number of documents to be returned. Cannot be used with 'maxChunkCount'. If maxDocumentCount is given, then only one chunk per document is returned.

validate_max_chunk_count_and_max_document_count()

max_chunk_count: int | None

max_document_count: int | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class GroundingType

Bases: str, Enum

Enumerates supported grounding types.

__new__(value)

DOCUMENT_GROUNDING_SERVICE = 'document_grounding_service'

class KeyValueListPair

Bases: ABCBaseModel

key: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

value: List[str]

gen_ai_hub.orchestration_v2.models.embeddings module

Embeddings Module Configuration Models

class EmbeddingResult

Bases: ABCBaseModel

A single embedding result.

Args:

object: The object type, always "embedding". embedding: The embedding vector (array of floats) or base64 string. index: The index of this embedding in the list.

embedding: List[float] | str

index: int

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

object: str

class EmbeddingsEncodingFormat

Bases: str, Enum

Encoding format for the embeddings output.

Values:

FLOAT: Returns embeddings as an array of floats. BASE64: Returns embeddings as a base64 encoded string. BINARY: Returns embeddings in binary format.

__new__(value)

BASE64 = 'base64'

BINARY = 'binary'

FLOAT = 'float'

class EmbeddingsInput

Bases: ABCBaseModel

Input for the embeddings endpoint.

Args:

text: The text to embed. Can be a single string or a list of strings. type: Optional type hint for the embedding model (text, document, or query).

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

text: str | List[str]

type_: EmbeddingsInputType | None

class EmbeddingsInputType

Bases: str, Enum

Type hint for the embedding model about the purpose of the text.

Some models use asymmetric embeddings for better search performance.

Values:

TEXT: General purpose text (default). DOCUMENT: Content to be searched/retrieved. QUERY: Short search queries.

__new__(value)

DOCUMENT = 'document'

QUERY = 'query'

TEXT = 'text'

class EmbeddingsModelConfig

Bases: ABCBaseModel

Configuration for the embeddings model.

Args:

model: The embedding model details.

model: EmbeddingsModelDetails

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class EmbeddingsModelDetails

Bases: ABCBaseModel

The model and parameters to be used for generating embeddings.

Args:

name: Name of the embedding model. version: Version of the model to be used. Defaults to "latest". params: Additional parameters for the model (dimensions, encoding_format, normalize). timeout: Timeout for the embeddings request in seconds. Ignored for Vertex AI models. max_retries: Maximum number of retries. Ignored for Vertex AI models.

max_retries: int | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str

params: EmbeddingsModelParams | None

timeout: int | None

version: str | None

class EmbeddingsModelParams

Bases: ABCBaseModel

Additional parameters for generating embeddings.

Args:

dimensions: The number of dimensions for the output embeddings. encoding_format: The format for the embeddings output (float, base64, or binary). normalize: Whether to normalize the embeddings.

dimensions: int | None

encoding_format: EmbeddingsEncodingFormat | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

normalize: bool | None

class EmbeddingsModuleConfigs

Bases: ABCBaseModel

Module configurations for the embeddings endpoint.

Args:

embeddings: Required configuration for the embeddings model. masking: Optional configuration for data masking before embedding.

embeddings: EmbeddingsModelConfig

masking: MaskingModuleConfig | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class EmbeddingsOrchestrationConfig

Bases: ABCBaseModel

Configuration for the Embeddings Orchestration endpoint.

Args:

modules: The module configurations including embeddings model and optional masking.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

modules: EmbeddingsModuleConfigs

class EmbeddingsPostResponse

Bases: ABCBaseModel

Response for an embeddings POST request.

Args:

request_id: Unique identifier for the request. intermediate_results: Optional results from intermediate modules (e.g., masking). final_result: The embeddings response from the model.

final_result: EmbeddingsResponse

intermediate_results: Dict | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str

class EmbeddingsRequest

Bases: ABCBaseModel

Request body for the embeddings endpoint.

Args:

config: The embeddings orchestration configuration. input: The input text to embed.

config: EmbeddingsOrchestrationConfig

input: EmbeddingsInput

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class EmbeddingsResponse

Bases: ABCBaseModel

The response from the embedding model, following OpenAI specification.

Args:

object: The object type, always "list". data: The list of embeddings generated by the model. model: The name of the model used to generate the embeddings. usage: Token usage information.

data: List[EmbeddingResult]

model: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

object: str

usage: EmbeddingsUsage

class EmbeddingsUsage

Bases: ABCBaseModel

Token usage information for the embeddings request.

Args:

prompt_tokens: The number of tokens used by the prompt. total_tokens: The total number of tokens used by the request.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

prompt_tokens: int

total_tokens: int

gen_ai_hub.orchestration_v2.models.llama_guard_3_filter module

Llama Guard 3 Content Filter Model

class LlamaGuard38bFilter

Bases: ABCBaseModel

Specific implementation of ContentFilter for Llama Guard 3. Llama Guard 3 is a Llama-3.1-8B pretrained model, fine-tuned for content safety classification.

Args:

violent_crimes: Responses that enable, encourage, or endorse the commission of violent crimes.

non_violent_crimes: Responses that enable, encourage, or endorse the commission of non-violent crimes.

sex_crimes: Responses that enable, encourage, or endorse the commission of sex-related crimes.

child_exploitation: Responses that contain, describe, enable or endorse the sexual abuse of children.

defamation: Responses that are both verifiably false and likely to injure a living person's reputation.

specialized_advice: Responses that contain specialized financial, medical or legal advice.

privacy: Responses that contain sensitive or nonpublic personal information.

intellectual_property: Responses that may violate the intellectual property rights of any third party.

indiscriminate_weapons: Responses that enable, encourage, or endorse the creation of indiscriminate weapons.

hate: Responses that demean or dehumanize people on the basis of their sensitive, personal characteristics.

self_harm: Responses that enable, encourage, or endorse acts of intentional self-harm.

sexual_content: Responses that contain erotica.

elections: Responses that contain factually incorrect information about electoral systems and processes.

code_interpreter_abuse: Responses that seek to abuse code interpreters.

child_exploitation: bool

code_interpreter_abuse: bool

defamation: bool

elections: bool

hate: bool

indiscriminate_weapons: bool

intellectual_property: bool

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

non_violent_crimes: bool

privacy: bool

self_harm: bool

sex_crimes: bool

sexual_content: bool

specialized_advice: bool

violent_crimes: bool

gen_ai_hub.orchestration_v2.models.llm_model_details module

Module defining the LLM (Large Language Model) configuration model.

class LLMModelDetails

Bases: ABCBaseModel

The model and parameters to be used for the prompt templating. This is the model that will be used to generate the response.

Args:

name: Name of the model as in LLM Access configuration.

version: Version of the model to be used. Defaults to "latest".

params: Additional parameters for the model. Default values are used for mandatory parameters.

timeout: Timeout for the LLM request in seconds. This parameter is currently ignored for Vertex AI models.

max_retries: Maximum number of retries for the LLM request. This parameter is currently ignored for Vertex AI models.

max_retries: int | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str

params: Dict | None

timeout: int | None

version: str | None

gen_ai_hub.orchestration_v2.models.message module

Defines message-related models for LLM-based conversations, including system, user, assistant, tool, and developer messages.

class AssistantMessage

Bases: ABCBaseModel

Represents an assistant message in a prompt or conversation template.

Assistant messages typically contain responses or outputs from the AI model.

Args:

role: The role of the entity sending the message.

content: The text content of the assistant message.

refusal: A string indicating refusal reason.

tool_calls: A list of tool call objects.

content: str | List[TextPart] | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

refusal: str | None

role: Role

tool_calls: List[MessageToolCall] | None

class DeveloperChatMessage

Bases: ABCBaseModel

content: str | List[TextPart]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: Role

class FunctionCall

Bases: ABCBaseModel

Represents a function call with its name and arguments.

Attributes:

name: str

The name of the function to call.

arguments: str

The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function.

parse_arguments()

Parses the arguments string as JSON.

Returns:

A dictionary representing the parsed arguments.

Return type:

dict

arguments: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str

class MessageToolCall

Bases: ABCBaseModel

The tool calls generated by the model, such as function calls.

Attributes:

id: The ID of the tool call.

type: The type of the tool. Currently, only function is supported.

function: The function that the model called.

function: FunctionCall

id: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type: Literal['function']

class ResponseChatMessage

Bases: ABCBaseModel

Represents a response message in a conversation.

Args:

role: The role of the entity sending the message.

content: The text content of the assistant message.

refusal: A string indicating refusal reason.

tool_calls: A list of tool call objects.

content: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

refusal: str | None

role: Role

tool_calls: List[MessageToolCall] | None

class Role

Bases: str, Enum

Enumerates supported roles in LLM-based conversations.

This enum defines the standard roles used in interactions with Large Language Models (LLMs). These roles are generally used to structure the input and distinguish between different parts of the conversation.

Values:

USER: Represents the human user's input in the conversation.

SYSTEM: Represents system-level instructions or context setting for the LLM.

ASSISTANT: Represents the LLM's responses in the conversation.

TOOL: Represents a tool or function that the LLM can call.

DEVELOPER: Represents the developer's input or instructions in the conversation.

__new__(value)

ASSISTANT = 'assistant'

DEVELOPER = 'developer'

SYSTEM = 'system'

TOOL = 'tool'

USER = 'user'

class SystemMessage

Bases: ABCBaseModel

Represents a system message in a prompt or conversation template.

System messages typically provide context or instructions to the AI model.

Args:

role: The role of the entity sending the message.

content: The text content of the system message.

content: str | List[TextPart]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: Role

class ToolChatMessage

Bases: ABCBaseModel

content: str | List[TextPart]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: Role

tool_call_id: str

class UserMessage

Bases: ABCBaseModel

Represents a user message in a prompt or conversation template.

User messages typically contain queries or inputs from the user.

Args:

role: The role of the entity sending the message.

content: The message content, which may be plain text or a sequence of text and images.

classmethod content_validation(content)

Validates and maps the content field to the appropriate types.

content: str | TextPart | ImagePart | List[str | TextPart | ImagePart | ImageItem]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: Role

gen_ai_hub.orchestration_v2.models.multimodal_items module

Models for representing multimodal content parts, including text and images.

class ImageDetailLevel

Bases: Enum

Controls the resolution and detail level for image analysis.

Attributes:

AUTO: The model determines the detail level automatically.

LOW: The model uses a low-fidelity, faster version of the image.

HIGH: The model uses a high-fidelity version of the image.

AUTO = 'auto'

HIGH = 'high'

LOW = 'low'

class ImageItem

Bases: ABCBaseModel

Represents an image for use in multimodal messages.

Args:

url: The image location, specified as either a standard URL or a data URL.

• Standard URL example: 'https://example.com/image.png'

• Data URL example: 'data:image/png;base64,...'

detail: The image detail level for model processing.

Example:

# Using a standard URL img1 = ImageItem(url="https://example.com/image.png", detail=ImageDetailLevel.HIGH)

# Using a data URL img2 = ImageItem(url="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA...")

static from_file(file_path, mime_type=None, detail=None)

Create an ImageItem from a local image file.

Parameters:

• file_path (str) -- Path to the image file.

• mime_type (Optional[str], optional) -- Explicit MIME type (e.g., 'image/png'). If not provided, the MIME type will be guessed from the file extension.

• detail (Optional[ImageDetailLevel], optional) -- The image detail level for model processing.

Raises:

ValueError -- If the MIME type cannot be determined and is not provided.

Returns:

An ImageItem instance with the image data as a data URL.

Return type:

ImageItem

model_dump(*, mode='python', include=None, exclude=None, context=None, by_alias=True, exclude_unset=False, exclude_defaults=False, exclude_none=False, round_trip=False, warnings=True, fallback=None, serialize_as_any=False)

Dumps the model to a dictionary with default settings.

Parameters:

• mode (Literal['json', 'python'] | str)

• include (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None)

• exclude (set[int] | set[str] | Mapping[int, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | Mapping[str, set[int] | set[str] | Mapping[int, IncEx | bool] | Mapping[str, IncEx | bool] | bool] | None)

• context (Any | None)

• by_alias (bool)

• exclude_unset (bool)

• exclude_defaults (bool)

• exclude_none (bool)

• round_trip (bool)

• warnings (bool | Literal['none', 'warn', 'error'])

• fallback (Callable[[Any], Any] | None)

• serialize_as_any (bool)

Return type:

dict[str, Any]

detail: ImageDetailLevel | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

url: str | None

class ImagePart

Bases: ABCBaseModel

Represents an image segment within a multimodal content block.

Args:

image_url: An ImageUrl object containing the image's location and detail level.

type: The type identifier, defaulting to "image_url".

image_url: ImageUrl

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: Literal['image_url']

class ImageUrl

Bases: ABCBaseModel

A data structure holding the URL and detail level for an image.

Args:

url: The location of the image, as a standard or data URL.

detail: The processing detail level for the image.

detail: ImageDetailLevel | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

url: str

class TextPart

Bases: ABCBaseModel

Represents a text segment within a multimodal content block.

Args:

text: The string content of the text part.

type: The type identifier, defaulting to "text".

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

text: str

type_: Literal['text']

gen_ai_hub.orchestration_v2.models.orchestration_request module

class CompletionPostRequest

Bases: ABCBaseModel

Represents a request for the orchestration process, including configuration, template values, and message history.

validate_config_or_ref()

validates that exactly one of 'config' or 'config_ref' is provided.

Raises:

ValueError -- If neither or both 'config' and 'config_ref' are provided.

Returns:

The validated OrchestrationRequest instance.

Return type:

CompletionPostRequest

config: OrchestrationConfig | None

config_ref: CompletionRequestConfigurationReferenceByIdConfigRef | CompletionRequestConfigurationReferenceByNameScenarioVersionConfigRef | None

messages_history: List[SystemMessage | UserMessage | AssistantMessage | ToolChatMessage | DeveloperChatMessage | ResponseChatMessage] | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

placeholder_values: dict[str, str] | None

gen_ai_hub.orchestration_v2.models.response module

Response models for orchestration v2

class ChatCompletionTokenLogprob

Bases: ABCBaseModel

Represents a token in the message content along with its log probability and alternative top log probabilities.

Attributes:

token: The token.

logprob: The log probability of this token.

bytes: UTF-8 bytes of the token, if applicable.

top_logprobs: List of most likely tokens and their log probabilities

at this token position.

bytes: List[int] | None

logprob: float

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

token: str

top_logprobs: List[TopLogprob] | None

class ChoiceLogprobs

Bases: ABCBaseModel

Log probabilities for the choice.

content: List[ChatCompletionTokenLogprob] | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

refusal: List[ChatCompletionTokenLogprob] | None

class Citation

Bases: ABCBaseModel

Represents a citation with related metadata.

Attributes:

ref_id (Optional[int]): Unique identifier for inline citation title (str): The title of the citation. url (str): The URL of the citation. start_index (Optional[int]): The starting index position of the citation

in a referenced text.

end_index (Optional[int]): The ending index position of the citation in

a referenced text.

end_index: int | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

ref_id: int | None

start_index: int | None

title: str

url: str

class CompletionPostResponse

Bases: ABCBaseModel

Represents the response for a completion post request.

Attributes:

request_id (str): Unique identifier for the completion request.

intermediate_results (ModuleResults): Results from various modules executed during the processing.

final_result (LLMModuleResult): Output from LLM. Follows the OpenAI spec.

final_result: LLMModuleResult

intermediate_failures: List[SAPAPIError] | None

intermediate_results: ModuleResults

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str

class CompletionTokensDetails

Bases: ABCBaseModel

Breakdown of tokens used in a completion.

Attributes:

accepted_prediction_tokens (Optional[int]): When using Predicted Outputs, the number of tokens in the

prediction that appeared in the completion.

audio_tokens (Optional[int]): Audio input tokens generated by the model. reasoning_tokens (Optional[int]): Tokens generated by the model for reasoning. rejected_prediction_tokens (Optional[int]): When using Predicted Outputs, the number of tokens in the

prediction that did not appear in the completion. However, like reasoning tokens, these tokens are still counted in the total completion tokens for purposes of billing, output, and context window limits.

accepted_prediction_tokens: int | None

audio_tokens: int | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

reasoning_tokens: int | None

rejected_prediction_tokens: int | None

class ErrorResponse

Bases: ABCBaseModel

error: SAPAPIError | list[SAPAPIError]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class ErrorResponseStreaming

Bases: ABCBaseModel

error: SAPAPIErrorStreaming | list[SAPAPIErrorStreaming]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class GenericModuleResult

Bases: ABCBaseModel

Generic module result Args:

message: Some message created from the module. Example: Input to LLM is masked successfully.

data: Additional data object from the module

data: Any | None

message: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class LLMChoice

Bases: ABCBaseModel

Args:

index: Index of the choice

message: Message from the LLM

logprobs: Log probabilities for the choice

finish_reason: Reason the model stopped generating tokens.

• 'stop' if the model hit a natural stop point or a provided stop sequence,

• 'length' if the maximum token number was reached,

• 'content_filter' if content was omitted due to a filter enforced by the LLM model provider

  or the content filtering module

finish_reason: str

index: int

logprobs: ChoiceLogprobs | None

message: ResponseChatMessage

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class LLMModuleResult

Bases: ABCBaseModel

Output from LLM. Follows the OpenAI spec.

Attributes:

id: Unique identifier for the response.

object: Type of object returned (e.g., "chat.completion").

created: Unix timestamp of when the result was created.

model: The model name (e.g., "gpt-4o-mini").

system_fingerprint: Optional system fingerprint associated with the result.

choices: List of LLMChoice objects representing the output choices.

usage: TokenUsage object representing the token usage statistics.

citations: Optional list of citations associated with the response.

choices: List[LLMChoice]

citations: list[Citation] | None

created: int

id: str

model: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

object: str

system_fingerprint: str | None

usage: TokenUsage

class ModuleResults

Bases: ABCBaseModel

Represents the results of each module used in a processing pipeline.

Attributes:

grounding: Optional result from the grounding module.

templating: Optional list of chat messages resulting from the templating

module.

input_translation: Optional result from the input translation module.

input_masking: Optional result from the input masking module.

input_filtering: Optional result from the input filtering module.

output_filtering: Optional result from the output filtering module.

output_translation: Optional result from the output translation module.

llm: Optional result from an LLM-specific module.

output_unmasking: Optional list of choices from the output unmasking

module.

grounding: GenericModuleResult | None

input_filtering: GenericModuleResult | None

input_masking: GenericModuleResult | None

input_translation: GenericModuleResult | None

llm: LLMModuleResult | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

output_filtering: GenericModuleResult | None

output_translation: GenericModuleResult | None

output_unmasking: List[LLMChoice] | None

templating: List[SystemMessage | UserMessage | AssistantMessage | ToolChatMessage | DeveloperChatMessage | ResponseChatMessage] | None

class OrchestrationResponseWithRetries

Bases: CompletionPostResponse

Extended CompletionPostResponse that includes retry count information.

This is returned when using retry-enabled methods like run_with_retries().

Attributes:

retries: Number of retry attempts that were made to successfully complete this request.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

retries: int

class PromptTokensDetails

Bases: ABCBaseModel

Represents the details of prompt tokens used in a specific operation.

Attributes:

audio_tokens (Optional[int]): Audio input tokens present in the prompt. cached_tokens (Optional[int]): Cached tokens present in the prompt.

audio_tokens: int | None

cached_tokens: int | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class SAPAPIError

Bases: ABCBaseModel

Represents an error returned from an SAP API.

Attributes:

request_id (str): The unique identifier of the request associated

with the error.

code (int): The http error code.

message (str): A detailed message describing the error.

location (str): The location where the error occurred

intermediate_results (Optional[ModuleResults]): Optional attribute

to store any processing results if available or applicable.

code: int

headers: dict[str, str] | None

intermediate_results: ModuleResults | None

location: str

message: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str

class SAPAPIErrorStreaming

Bases: ABCBaseModel

Represents an error returned from an SAP API.

Attributes:

request_id (str): The unique identifier of the request associated

with the error.

code (int): The http error code.

message (str): A detailed message describing the error.

location (str): The location where the error occurred

intermediate_results (Optional[ModuleResults]): Optional attribute

to store any processing results if available or applicable.

code: int

headers: dict[str, str] | None

intermediate_results: ModuleResultsStreaming | None

location: str

message: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str

class StreamCompletionPostResponse

Bases: ABCBaseModel

final_result: StreamLLMModuleResult | None

intermediate_failures: List[SAPAPIError] | None

intermediate_results: StreamModuleResults | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

request_id: str

class StreamDelta

Bases: ABCBaseModel

content: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

role: str | None

tool_calls: List[StreamToolCall] | None

class StreamFunctionObject

Bases: FunctionCall

Represents a function call with its name and arguments.

Attributes:

name: str

The name of the function to call.

arguments: str

The arguments to call the function with, as generated by the model in JSON format. Note that the model does not always generate valid JSON, and may hallucinate parameters not defined by your function schema. Validate the arguments in your code before calling your function.

arguments: str | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str | None

class StreamLLMChoice

Bases: ABCBaseModel

delta: StreamDelta

finish_reason: str | None

index: int

logprobs: ChoiceLogprobs | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class StreamLLMModuleResult

Bases: LLMModuleResult

choices: List[StreamLLMChoice]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

usage: TokenUsage | None

class StreamModuleResults

Bases: ModuleResults

llm: StreamLLMModuleResult | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

output_unmasking: List[StreamLLMChoice] | None

class StreamToolCall

Bases: ABCBaseModel

function: StreamFunctionObject | None

id: str | None

index: int

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: Literal['function']

class TokenUsage

Bases: ABCBaseModel

Usage of tokens in the response

completion_tokens: int

completion_tokens_details: CompletionTokensDetails | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

prompt_tokens: int

prompt_tokens_details: PromptTokensDetails | None

total_tokens: int

class TopLogprob

Bases: ABCBaseModel

Represents one of the most likely tokens and its log probability at a given token position.

Attributes:

token: The token.

logprob: The log probability of this token.

bytes: UTF-8 bytes of the token, if applicable.

bytes: List[int] | None

logprob: float

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

token: str

gen_ai_hub.orchestration_v2.models.response_format module

Response format models for model output specification.

class JSONResponseSchema

Bases: ABCBaseModel

Response format JSON Schema that the model output should adhere to.

Args:

name: The name of the response format.

description: A description of what the response format is for.

schema: A schema for the response format described as a JSON Schema object.

strict: Whether to enable strict schema adherence when generating the output.

classmethod validate_name(name)

validates the name of the response format.

Parameters:

name (str) -- the name to validate

Raises:

ValueError -- if the name does not match the required pattern or exceeds the maximum length

Returns:

the validated name

Return type:

str

description: str | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str

schema_: dict

strict: bool

class ResponseFormatJsonObject

Bases: ABCBaseModel

Response format JSON Object that the model output should adhere to.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ResponseFormatType

class ResponseFormatJsonSchema

Bases: ABCBaseModel

json_schema: JSONResponseSchema

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ResponseFormatType

class ResponseFormatText

Bases: ABCBaseModel

Response format that the model output should adhere to.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: ResponseFormatType

class ResponseFormatType

Bases: str, Enum

Enumerates the supported response format.

Response format that the model output should adhere to. This is the same as the OpenAI definition.

Values:

TEXT: Response format as text

JSON_OBJECT: Response format as json object

JSON_SCHEMA: Response format as defined json schema

__new__(value)

JSON_OBJECT = 'json_object'

JSON_SCHEMA = 'json_schema'

TEXT = 'text'

gen_ai_hub.orchestration_v2.models.streaming module

Streaming options for content generation.

class GlobalStreamOptions

Bases: ABCBaseModel

Represents options for streaming content generation. Args:

enabled(bool, optional): If true, the response will be streamed back to the client.

chunk_size(int, optional): Minimum number of characters per chunk that post-LLM modules operate on.

delimiters(list(str), optional): List of delimiters to split the input text into chunks.Please note, this is a required parameter when input_translation_module_config or output_translation_module_config are configured.

model_dump(**kwargs)

Override model_dump to exclude chunk_size and delimiters when enabled is False.

chunk_size: int | None

delimiters: List[str] | None

enabled: bool | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

gen_ai_hub.orchestration_v2.models.template module

Defines data models for prompt templating in the orchestration module.

class PromptTemplatingModuleConfig

Bases: ABCBaseModel

model: LLMModelDetails

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

prompt: Template | TemplateRef

class Template

Bases: ABCBaseModel

Represents a configurable template for generating prompts or conversations.

Args:

template: A list of prompt messages that form the template.

defaults: A dict of default values for template variables.

tools: A list of tool definitions.

response_format: A response format that the model output should adhere to.

defaults: dict | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

response_format: ResponseFormatText | ResponseFormatJsonObject | ResponseFormatJsonSchema | None

template: List[SystemMessage | UserMessage | AssistantMessage | ToolChatMessage | DeveloperChatMessage | ResponseChatMessage]

tools: List[dict | FunctionTool] | None

gen_ai_hub.orchestration_v2.models.template_ref module

Module for template reference models.

class TemplateRef

Bases: ABCBaseModel

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

template_ref: TemplateRefByID | TemplateRefByScenarioNameVersion

class TemplateRefByID

Bases: ABCBaseModel

Represents a prompt template reference for generating prompts or conversations. Args:

id(str): ID of the template in prompt registry scope(Optional[Literal["resource_group", "tenant"]]): Defines the scope that is searched

for the referenced template. 'tenant' indicates the template is shared across all resource groups within the tenant, while 'resource_group' indicates the template is only accessible within the specific resource group. Defaults to 'tenant'.

id: str

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

scope: Literal['resource_group', 'tenant'] | None

class TemplateRefByScenarioNameVersion

Bases: ABCBaseModel

Represents a prompt template reference for generating prompts or conversations. Args:

scenario(str): Scenario name

name(str): Name of template

version(str): Version of template

scope(Optional[Literal["resource_group", "tenant"]]): Defines the scope that is searched

for the referenced template. 'tenant' indicates the template is shared across all resource groups within the tenant, while 'resource_group' indicates the template is only accessible within the specific resource group. Defaults to 'tenant'.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str

scenario: str

scope: Literal['resource_group', 'tenant'] | None

version: str

gen_ai_hub.orchestration_v2.models.tools module

Models for tools used in chat completions with function calling.

class ChatCompletionTool

Bases: ABCBaseModel

Base class for all chat completion tools.

Args:

type (Literal["function"]): The type of the tool. Currently, only function is supported.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: Literal['function']

class FunctionObject

Bases: ABCBaseModel

Represents a function. Args:

name (str): The name of the function to be called. Must be a-z, A-Z, 0-9,

or contain underscores and dashes, with a maximum length of 64.

description (str): A description of what the function does, used by the model

to choose when and how to call the function.

parameters (dict): The parameters the functions accepts, described as a JSON Schema object.

Omitting parameters defines a function with an empty parameter list.

strict (bool, optional): Whether to enable strict schema adherence when generating the function call.

If set to true, the model will follow the exact schema defined in the parameters field. Only a subset of JSON Schema is supported when strict is true. Defaults to False.

description: str | None

function: Callable | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

name: str

parameters: dict | None

strict: bool

class FunctionTool

Bases: ChatCompletionTool

Represents a function tool for OpenAI-like function calling.

Args:

type (Literal["function"]): The type of the tool. Currently, only function is supported.

function (FunctionObject): The function to be called.

static from_function(func, *, description=None, strict=False)

Create a FunctionTool from a Python function.

Args:

func (Callable): The function to be converted to a FunctionTool.

description (Optional[str]): A description of the function. Defaults to the docstring of the function.

strict (bool): Whether to enable strict schema adherence when generating the function call.

Parameters:

• func (Callable)

• description (str | None)

• strict (bool)

Return type:

FunctionTool

async aexecute(**kwargs)

Asynchronously execute the function with the provided arguments.

Parameters:

kwargs (Any)

Return type:

Any

execute(**kwargs)

Execute the function with the provided arguments.

Parameters:

kwargs (Any)

Return type:

Any

function: FunctionObject

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: Literal['function']

function_tool(func=None, *, description=None, strict=False)

Decorator that converts a function into a FunctionTool.

Usage:

@function_tool def my_func(...): ...

@function_tool() def my_func(...): ...

Parameters:

• func (Callable | None)

• description (str | None)

• strict (bool)

Return type:

Callable[[Callable], FunctionTool] | FunctionTool

python_type_to_json_type(py_type)

Convert a Python type to a JSON Schema type.

Parameters:

py_type (any) -- the Python type to convert

Returns:

A dictionary representing the JSON Schema type.

Return type:

dict

gen_ai_hub.orchestration_v2.models.translation module

Translation module configuration models.

class InputTranslationConfig

Bases: TranslationConfig

Configuration for input translation.

Args:

source_language: Language of the text to be translated. Example: de-DE target_language: Language to which the text should be translated. Example: en-US apply_to: List of selectors that define the scope of translation.

apply_to: list[SAPDocumentTranslationApplyToSelector] | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class OutputTranslationConfig

Bases: TranslationConfig

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

target_language: str | SAPDocumentTranslationApplyToSelector

class SAPDocumentTranslation

Bases: ABCBaseModel

Configuration for translation module.

Args:

type: The type of translation module (e.g., 'sap_document_translation').

config: Configuration object for the translation module.

config: TranslationConfig

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

type_: TranslationType

class SAPDocumentTranslationApplyToSelector

Bases: ABCBaseModel

This selector allows you to define the scope of translation, such as specific placeholders or messages with specific roles. For example, {"category": "placeholders",

"items": ["user_input"], "source_language": "de-DE"} targets the value of "user_input" in placeholder_values specified in the request payload; and considers the value to be in German.

category: Literal['placeholders', 'template_roles']

items: list[str]

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

source_language: str

class SAPDocumentTranslationInput

Bases: SAPDocumentTranslation

Configuration for input translation

Args:

type: The type of translation module (e.g., 'sap_document_translation').

translate_messages_history: If true, the messages history will be translated as well.

config: Configuration object for the translation module.

config: InputTranslationConfig | TranslationConfig

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

translate_messages_history: bool | None

class SAPDocumentTranslationOutput

Bases: SAPDocumentTranslation

Configuration for output translation

Args:

type: The type of translation module (e.g., 'sap_document_translation').

config: Configuration object for the translation module.

config: OutputTranslationConfig | TranslationConfig

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class TranslationConfig

Bases: ABCBaseModel

Configuration for sap_document_translation translation provider.

Args:

source_language: Language of the text to be translated. Example: de-DE

target_language: Language to which the text should be translated. Example: en-US

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

source_language: str | None

target_language: str

class TranslationModuleConfig

Bases: ABCBaseModel

Configuration for translation module

Args:

input: Configuration for input translation

output: Configuration for output translation

input: SAPDocumentTranslationInput | SAPDocumentTranslation | None

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': False}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

output: SAPDocumentTranslationOutput | SAPDocumentTranslation | None

class TranslationType

Bases: str, Enum

Enumerates supported translation types.

__new__(value)

SAP_DOCUMENT_TRANSLATION = 'sap_document_translation'