gen_ai_hub.orchestration.models package

Submodules

gen_ai_hub.orchestration.models.azure_content_filter module

class AzureContentFilter

Bases: ContentFilter

Specific implementation of ContentFilter for Azure's content filtering service.

This class configures content filtering based on Azure's categories and severity levels. It allows setting thresholds for hate speech, sexual content, violence, and self-harm content.

__init__(hate, sexual, violence, self_harm, **kwargs)

Initializes the AzureContentFilter with specified thresholds for different content categories.

Parameters:

  • hate (Union[AzureThreshold, Literal[0, 2, 4, 6]]) -- threshold for hate speech content

  • sexual (Union[AzureThreshold, Literal[0, 2, 4, 6]]) -- threshold for sexual content

  • violence (Union[AzureThreshold, Literal[0, 2, 4, 6]]) -- threshold for violent content

  • self_harm (Union[AzureThreshold, Literal[0, 2, 4, 6]]) -- threshold for self-harm content

class AzureThreshold

Bases: int, Enum

Enumerates the threshold levels for the Azure Content Safety service.

This enum defines the various threshold levels that can be used to filter content based on its safety score. Each threshold value represents a specific level of content moderation.

Values:

ALLOW_SAFE: Allows only Safe content.

ALLOW_SAFE_LOW: Allows Safe and Low content.

ALLOW_SAFE_LOW_MEDIUM: Allows Safe, Low, and Medium content.

ALLOW_ALL: Allows all content (Safe, Low, Medium, and High).

__new__(value)
ALLOW_ALL = 6
ALLOW_SAFE = 0
ALLOW_SAFE_LOW = 2
ALLOW_SAFE_LOW_MEDIUM = 4

gen_ai_hub.orchestration.models.base module

class JSONSerializable

Bases: ABC

An interface for objects that can be serialized to JSON.

abstractmethod to_dict()

Convert the object to a JSON-serializable dictionary.

Returns:

dictionary representation of the object.

Return type:

Dict[str, Any]

gen_ai_hub.orchestration.models.config module

class OrchestrationConfig

Bases: JSONSerializable

Configuration for the Orchestration Service's content generation process.

Defines modules for a harmonized API that combines LLM-based content generation with additional processing functionalities.

The orchestration service allows for advanced content generation by processing inputs through a series of steps: template rendering, text generation via LLMs, and optional input/output transformations such as data masking or filtering.

__init__(template, llm, filtering=None, data_masking=None, grounding=None, stream_options=None, translation=None)

Initializes the OrchestrationConfig with specified modules.

Parameters:

  • template (Union[Template, TemplateRef]) -- template for rendering input prompts

  • llm (LLM) -- language model for text generation

  • filtering (Optional[ContentFiltering], optional) -- content filtering module, defaults to None

  • data_masking (Optional[DataMasking], optional) -- data masking module, defaults to None

  • grounding (Optional[GroundingModule], optional) -- document grounding module, defaults to None

  • stream_options (Optional[dict], optional) -- global streaming options, defaults to None

  • translation (Optional[Translation], optional) -- translation module, defaults to None

to_dict()

Converts the orchestration configuration to a dictionary format.

Returns:

dictionary representation of the orchestration configuration.

Return type:

dict

gen_ai_hub.orchestration.models.content_filter module

class ContentFilter

Bases: JSONSerializable

Base class for content filtering configurations.

This class provides a generic structure for defining content filters from various providers. It allows for specifying the provider and associated configuration parameters.

__init__(provider, config)

Initializes the ContentFilter with specified provider and configuration.

Parameters:

  • provider (Union[ContentFilterProvider, str]) -- The name of the content filter provider.

  • config (dict) -- A dictionary containing the configuration parameters for the content filter.

to_dict()

to_dict method to convert the content filter to a dictionary.

Returns:

dictionary representation of the content filter.

Return type:

dict

class ContentFilterProvider

Bases: str, Enum

Enumerates supported content filter providers.

This enum defines the available content filtering services that can be used for content moderation tasks. Each enum value represents a specific provider.

Values:

AZURE: Represents the Azure Content Safety service.

LLAMA_GUARD_3_8B: Represents the Llama Guard 3 based on Llama-3.1-8B pretrained model.

__new__(value)
AZURE = 'azure_content_safety'
LLAMA_GUARD_3_8B = 'llama_guard_3_8b'

gen_ai_hub.orchestration.models.content_filtering module

class ContentFiltering

Bases: JSONSerializable

Module for managing and applying content filters.

__init__(input_filtering=None, output_filtering=None)

Initializes the ContentFiltering with optional input and output filtering configurations.

Parameters:

  • input_filtering (Optional[InputFiltering], optional) -- the configuration for input filtering, defaults to None

  • output_filtering (Optional[OutputFiltering], optional) -- the configuration for output filtering, defaults to None

to_dict()

to_dict method to convert the content filtering configuration to a dictionary.

Returns:

dictionary representation of the content filtering configuration.

Return type:

dict

class InputFiltering

Bases: JSONSerializable

Module for managing and applying input content filters.

__init__(filters)

Initializes the InputFiltering with specified filters.

Parameters:

filters (List[ContentFilter]) -- List of ContentFilter objects to be applied to input content.

to_dict()

to_dict method to convert the input filtering configuration to a dictionary.

Returns:

dictionary representation of the input filtering configuration.

Return type:

dict

class OutputFiltering

Bases: JSONSerializable

Module for managing and applying output content filters.

__init__(filters, stream_options=None)

Initializes the OutputFiltering with specified filters and optional streaming options.

Parameters:

  • filters (List[ContentFilter]) -- List of ContentFilter objects to be applied to output content.

  • stream_options (Optional[dict], optional) -- Module-specific streaming options, defaults to None

to_dict()

to_dict method to convert the output filtering configuration to a dictionary.

Returns:

dictionary representation of the output filtering configuration.

Return type:

dict

gen_ai_hub.orchestration.models.data_masking module

class DataMasking

Bases: JSONSerializable

Manages data masking operations using the specified providers.

The DataMasking class is responsible for configuring and executing data masking processes by delegating to one or more data masking providers. It supports either anonymization or pseudonymization of sensitive information, depending on the provider and method used.

__init__(providers)

Initializes the DataMasking instance with the specified providers.

Parameters:

providers (List[DataMaskingProvider]) -- A list of data masking providers.

Raises:

ValueError -- If more than one provider is specified, as multiple providers are not supported in the current version.

to_dict()

Converts the DataMasking instance to a dictionary representation.

Returns:

A dictionary containing the data masking providers.

Return type:

dict

class DataMaskingProvider

Bases: JSONSerializable, ABC

Abstract base class for data masking providers.

This class serves as a blueprint for implementing different data masking providers. Each provider is responsible for masking sensitive or personally identifiable information (PII) according to a specific method.

Inherited by:

  • SAPDataPrivacyIntegration

class DataMaskingProviderName

Bases: str, Enum

Enumerates the available data masking providers.

This enum defines the supported providers for masking sensitive data in the LLM module.

Values: SAP_DATA_PRIVACY_INTEGRATION: Refers to the SAP Data Privacy Integration service, which offers anonymization and pseudonymization capabilities for sensitive data.

__new__(value)
SAP_DATA_PRIVACY_INTEGRATION = 'sap_data_privacy_integration'

gen_ai_hub.orchestration.models.document_grounding module

class DataRepositoryType

Bases: str, Enum

Enumerates data repository types.

__new__(value)
URL = 'help.sap.com'
VECTOR = 'vector'
class DocumentGrounding

Bases: JSONSerializable

defines the detailed configuration for the Grounding module.

__init__(input_params, output_param, filters=None, metadata_params=None)

Initializes the DocumentGrounding instance.

Parameters:

  • input_params (List[str]) -- The list of input parameters used for grounding input questions.

  • output_param (str) -- Parameter name used for grounding output.

  • filters (List[DocumentGroundingFilter], optional) -- List of DocumentGroundingFilter objects, defaults to None

  • metadata_params (List[str], optional) -- Parameter name used for specifying metadata parameters, defaults to None

to_dict()

Converts the DocumentGrounding instance to a dictionary representation.

Returns:

Dictionary representation of the DocumentGrounding.

Return type:

dict

class DocumentGroundingFilter

Bases: JSONSerializable

Module for configuring document grounding filters.

__init__(id, data_repository_type, search_config=None, data_repositories=None, data_repository_metadata=None, document_metadata=None, chunk_metadata=None)

Initializes the DocumentGroundingFilter instance.

Parameters:

  • id (str) -- The unique identifier for the grounding filter.

  • data_repository_type (str) -- Only include DataRepositories with the given type: 'vector' or 'url' of website supporting elastic search.

  • search_config (GroundingFilterSearch, optional) -- GroundingFilterSearchConfiguration object, defaults to None

  • data_repositories (List[str], optional) -- list of data repositories to search. Specify ['*'] to search across all DataRepositories or give a specific list of DataRepository ids, defaults to None

  • data_repository_metadata (List[Dict[str, Any]], optional) -- The metadata for the data repository, Restrict DataRepositories considered during search to those annotated with the given metadata. Useful when combined with dataRepositories=['*'], defaults to None

  • document_metadata (List[DocumentMetadata], optional) -- DocumentMetadata object, defaults to None

  • chunk_metadata (List[Dict[str, Any]], optional) -- Restrict chunks considered during search to those with the given metadata, defaults to None

to_dict()

Converts the DocumentGroundingFilter instance to a dictionary representation.

Returns:

Dictionary representation of the DocumentGroundingFilter.

Return type:

dict

class DocumentMetadata

Bases: JSONSerializable

Restrict documents considered during search to those annotated with the given metadata.

__init__(key, value, select_mode=None)

Initializes the DocumentMetadata instance.

Parameters:

  • key (str) -- The key for the metadata.

  • value (List[str]) -- The list of values for the metadata.

  • select_mode (List[str], optional) -- Select mode for search filters.

to_dict()

Converts the DocumentMetadata instance to a dictionary representation.

Returns:

Dictionary representation of the DocumentMetadata.

Return type:

dict

class GroundingFilterSearch

Bases: JSONSerializable

Search configuration for the data repository.

__init__(max_chunk_count=None, max_document_count=None)

Initializes the GroundingFilterSearch instance.

Parameters:

  • max_chunk_count (int, optional) -- maximum number of chunks > 0 to return, defaults to None

  • max_document_count (int, optional) -- Maximum number of documents > 0 to return. Only supports 'vector' dataRepositoryType. Cannot be used with 'maxChunkCount'. If maxDocumentCount is given, then only one chunk per document is returned, defaults to None

Raises:

ValueError -- If both max_chunk_count and max_document_count are set.

to_dict()

Converts the GroundingFilterSearch instance to a dictionary representation.

Returns:

Dictionary representation of the GroundingFilterSearch.

Return type:

dict

class GroundingModule

Bases: JSONSerializable

Module for managing and applying grounding aka RAG configurations.

__init__(type, config)

Initializes the GroundingModule instance.

Parameters:

  • type (str) -- The type of the grounding module.

  • config (DocumentGrounding) -- Configuration for the grounding module.

to_dict()

Converts the GroundingModule instance to a dictionary representation.

Returns:

Dictionary representation of the GroundingModule.

Return type:

dict

class GroundingType

Bases: str, Enum

Enumerates supported grounding types.

__new__(value)
DOCUMENT_GROUNDING_SERVICE = 'document_grounding_service'

gen_ai_hub.orchestration.models.llama_guard_3_filter module

class LlamaGuard38bFilter

Bases: ContentFilter

Specific implementation of ContentFilter for Llama Guard 3. Llama Guard 3 is a Llama-3.1-8B pretrained model, fine-tuned for content safety classification.

__init__(violent_crimes=False, non_violent_crimes=False, sex_crimes=False, child_exploitation=False, defamation=False, specialized_advice=False, privacy=False, intellectual_property=False, indiscriminate_weapons=False, hate=False, self_harm=False, sexual_content=False, elections=False, code_interpreter_abuse=False, **kwargs)

Initializes the LlamaGuard38bFilter with specified content filtering options.

Parameters:

  • violent_crimes (bool, optional) -- Responses that enable, encourage, or endorse the commission of violent crimes. Defaults to False.

  • non_violent_crimes (bool, optional) -- Responses that enable, encourage, or endorse the commission of non-violent crimes. Defaults to False.

  • sex_crimes (bool, optional) -- Responses that enable, encourage, or endorse the commission of sex-related crimes. Defaults to False.

  • child_exploitation (bool, optional) -- Responses that contain, describe, enable or endorse the sexual abuse of children. Defaults to False.

  • defamation (bool, optional) -- Responses that are both verifiably false and likely to injure a living person's reputation. Defaults to False.

  • specialized_advice (bool, optional) -- Responses that contain specialized financial, medical or legal advice. Defaults to False.

  • privacy (bool, optional) -- Responses that contain sensitive or nonpublic personal information. Defaults to False.

  • intellectual_property (bool, optional) -- Responses that may violate the intellectual property rights of any third party. Defaults to False.

  • indiscriminate_weapons (bool, optional) -- Responses that enable, encourage, or endorse the creation of indiscriminate weapons. Defaults to False.

  • hate (bool, optional) -- Responses that demean or dehumanize people on the basis of their sensitive, personal characteristics. Defaults to False.

  • self_harm (bool, optional) -- Responses that enable, encourage, or endorse acts of intentional self-harm. Defaults to False.

  • sexual_content (bool, optional) -- Responses that contain erotica. Defaults to False.

  • elections (bool, optional) -- Responses that contain factually incorrect information about electoral systems and processes. Defaults to False.

  • code_interpreter_abuse (bool, optional) -- Responses that seek to abuse code interpreters. Defaults to False.

gen_ai_hub.orchestration.models.llm module

class LLM

Bases: JSONSerializable

Represents a Large Language Model (LLM) configuration.

This class encapsulates the details required to specify and configure a particular LLM for use in natural language processing tasks. It includes the model's name, version, and any additional parameters needed for its operation.

__init__(name, version='latest', parameters=None)

Initializes the LLM with specified name, version, and parameters.

Parameters:

  • name (str) -- Name of the LLM.

  • version (str, optional) -- Version of the LLM, defaults to "latest"

  • parameters (Optional[Dict], optional) --

    Additional parameters for the LLM, defaults to None

    Common parameters include:

    • 'temperature': Controls randomness in output. Lower values (e.g., 0.2) make output more focused and deterministic, while higher values (e.g., 0.8) make output more diverse and creative.

    • 'max_tokens': Sets the maximum number of tokens to generate in the response. This can help control the length of the model's output.

to_dict()

Converts the LLM instance to a dictionary representation.

Returns:

Dictionary representation of the LLM.

Return type:

dict

gen_ai_hub.orchestration.models.message module

class AssistantMessage

Bases: Message

Represents an assistant message in a prompt or conversation template.

Assistant messages typically contain responses or outputs from the AI model.

__init__(content, refusal=None, tool_calls=None)

Initializes an AssistantMessage instance.

Parameters:

  • content (str) -- The text content of the assistant message.

  • refusal (Optional[str], optional) -- A string indicating refusal reason, defaults to None

  • tool_calls (Optional[List[MessageToolCall]], optional) -- A list of tool call objects, defaults to None

class FunctionCall

Bases: object

The function that the model called.

__init__(name=None, arguments=None)
Parameters:

  • name (str | None)

  • arguments (str | None)

Return type:

None

parse_arguments()

Parses the arguments string as JSON.

Returns:

A dictionary representing the parsed arguments.

Return type:

dict

arguments: str | None = None
name: str | None = None
class Message

Bases: JSONSerializable

Represents a single message in a prompt or conversation template.

This base class defines the structure for all types of messages in a prompt, including content and role.

Args:

role: The role of the entity sending the message.

content: The message content, which may be plain text or a sequence of text and images.

__init__(role, content, refusal=None, tool_calls=None)
Parameters:
Return type:

None

to_dict()

Converts the Message instance to a dictionary.

Returns:

A dictionary representation of the Message instance.

Return type:

dict

content: str | List[TextPart | ImagePart]
refusal: str | None = None
role: Role | str
tool_calls: List[MessageToolCall] | None = None
class MessageToolCall

Bases: object

Represents a tool call within a message, specifically a function call.

__init__(id, type, function)
Parameters:

  • id (str)

  • type (Literal['function'])

  • function (FunctionCall)

Return type:

None

to_dict()

Converts the MessageToolCall instance to a dictionary.

Returns:

A dictionary representation of the MessageToolCall instance.

Return type:

dict

function: FunctionCall
id: str
type: Literal['function']
class Role

Bases: str, Enum

Enumerates supported roles in LLM-based conversations.

This enum defines the standard roles used in interactions with Large Language Models (LLMs). These roles are generally used to structure the input and distinguish between different parts of the conversation.

Values:

  • USER: Represents the human user's input in the conversation.

  • SYSTEM: Represents system-level instructions or context setting for the LLM.

  • ASSISTANT: Represents the LLM's responses in the conversation.

  • TOOL: Represents a tool or function that the LLM can call.

  • DEVELOPER: Represents the developer's input or instructions in the conversation.

__new__(value)
ASSISTANT = 'assistant'
DEVELOPER = 'developer'
SYSTEM = 'system'
TOOL = 'tool'
USER = 'user'
class SystemMessage

Bases: Message

Represents a system message in a prompt or conversation template.

System messages typically provide context or instructions to the AI model.

__init__(content)

Initializes a SystemMessage instance.

Parameters:

content (str) -- The text content of the system message.

class ToolMessage

Bases: Message

Represents a tool message in a prompt or conversation template.

Parameters:

Message (str) -- The text content of the tool message.

__init__(content, tool_call_id)

Initializes a ToolMessage instance.

Parameters:

  • content (str) -- The text content of the tool message.

  • tool_call_id (str) -- The ID of the tool call associated with this message.

to_dict()

Converts the ToolMessage instance to a dictionary.

Returns:

A dictionary representation of the ToolMessage instance.

Return type:

dict

class UserMessage

Bases: Message

Represents a user message in a prompt or conversation template.

User messages typically contain queries or inputs from the user.

__init__(content)

Initializes a UserMessage instance.

Parameters:

content (Union[str, List[Union[str, ImageItem]]]) -- The message content, which may be plain text or a sequence of text and images.

Raises:

TypeError -- If the content list contains unsupported types.

gen_ai_hub.orchestration.models.multimodal_items module

class ImageDetailLevel

Bases: Enum

Controls the resolution and detail level for image analysis.

Attributes:

  • AUTO: The model determines the detail level automatically.

  • LOW: The model uses a low-fidelity, faster version of the image.

  • HIGH: The model uses a high-fidelity version of the image.

AUTO = 'auto'
HIGH = 'high'
LOW = 'low'
class ImageItem

Bases: JSONSerializable

Represents an image for use in multimodal messages.

Examples:

Using a standard URL img1 = ImageItem(url="https://example.com/image.png", detail=ImageDetailLevel.HIGH)

Using a data URL img2 = ImageItem(url="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA...")

static from_file(file_path, mime_type=None, detail=None)

Creates an ImageItem from a local image file.

Parameters:

  • file_path (str) -- Path to the image file.

  • mime_type (Optional[str], optional) -- Explicit MIME type (e.g., 'image/png'). If not provided, the MIME type will be guessed from the file extension.

  • detail (Optional[ImageDetailLevel], optional) -- The image detail level for model processing.

Raises:

  • ValueError -- If the MIME type cannot be determined and is not provided.

  • FileNotFoundError -- If the file does not exist.

Returns:

An ImageItem instance with the image data as a data URL.

Return type:

ImageItem

__init__(url=None, detail=None)

Initializes an ImageItem instance.

Parameters:
to_dict()

Converts the ImageItem instance to a dictionary representation.

Returns:

A dictionary representation of the ImageItem.

Return type:

Dict[str, Any]

class ImagePart

Bases: JSONSerializable

Represents an image segment within a multimodal content block.

Args:

  • image_url: An ImageUrl object containing the image's location and detail level.

  • type: The type identifier, defaulting to "image_url".

__init__(image_url, type='image_url')
Parameters:
Return type:

None

to_dict()

Converts the ImagePart instance to a dictionary.

Returns:

A dictionary representation of the ImagePart.

Return type:

dict

image_url: ImageUrl
type: str = 'image_url'
class ImageUrl

Bases: object

A data structure holding the URL and detail level for an image.

Args:

  • url: The location of the image, as a standard or data URL.

  • detail: The processing detail level for the image.

__init__(url, detail=None)
Parameters:
Return type:

None

detail: ImageDetailLevel | None = None
url: str
class TextPart

Bases: JSONSerializable

Represents a text segment within a multimodal content block.

Args:

  • text: The string content of the text part.

  • type: The type identifier, defaulting to "text".

__init__(text, type='text')
Parameters:

  • text (str)

  • type (str)

Return type:

None

to_dict()

Converts the TextPart instance to a dictionary.

Returns:

A dictionary representation of the TextPart.

Return type:

dict

text: str
type: str = 'text'

gen_ai_hub.orchestration.models.response module

class BaseLLMResult

Bases: object

Base class for LLM results containing common attributes.

Attributes:

id: Unique identifier for the LLM operation. object: Type of object returned (e.g., "chat.completion"). created: Timestamp when this result was created. model: Name or identifier of the model used.

__init__(id, object, created, model)
Parameters:

  • id (str)

  • object (str)

  • created (int)

  • model (str)

Return type:

None

created: int
id: str
model: str
object: str
class BaseModuleResults

Bases: object

Base class for module results containing grounding, common filtering and masking attributes.

Attributes:

input_filtering: Results from the input filtering module.

output_filtering: Results from the output filtering module.

input_masking: Results from the input masking module.

grounding: A list of extracted text to be provided as grounding context.

input_translation: Results from the input translation module.

output_translation: Results from the output translation module.

__init__(input_filtering=None, output_filtering=None, input_masking=None, grounding=None, input_translation=None, output_translation=None)
Parameters:
Return type:

None

grounding: GenericModuleResult | None = None

A list of extracted text to be provided as grounding context.

input_filtering: GenericModuleResult | None = None

Results from the input filtering module.

input_masking: GenericModuleResult | None = None

Results from the input masking module.

input_translation: GenericModuleResult | None = None

Results from the input translation module.

output_filtering: GenericModuleResult | None = None

Results from the output filtering module.

output_translation: GenericModuleResult | None = None

Results from the output translation module.

class ChatDelta

Bases: object

Represents a partial update in a streaming chat response.

__init__(content, role=None, refusal=None, tool_calls=None)
Parameters:
Return type:

None

content: str | List[TextPart | ImagePart]

The text content of the chat delta.

refusal: str | None = None

Optional refusal reason if the model refused to generate content.

role: str | None = None

Optional role identifier (e.g., 'assistant', 'user') for the message delta.

tool_calls: List[ToolCallChunk] | None = None

Optional list of tool call chunks associated with this chat delta.

class GenericModuleResult

Bases: object

Represents a generic module result in the orchestration process.

__init__(message, data=None)
Parameters:

  • message (str)

  • data (Dict[str, Any] | None)

Return type:

None

data: Dict[str, Any] | None = None

Additional data relevant to the module result.

message: str

A message or description generated by the module.

class LLMChoice

Bases: object

Represents an individual choice or response generated by the LLM.

Attributes:

index: The index of this particular choice in the list of possible choices.

message: The message object containing the role and content of the response.

finish_reason: The reason why the model stopped generating tokens.

logprobs: Optional dictionary containing token log probabilities.

__init__(index, message, finish_reason, logprobs=None)
Parameters:

  • index (int)

  • message (Message)

  • finish_reason (str)

  • logprobs (Dict[str, float] | None)

Return type:

None

finish_reason: str

The reason why the model stopped generating tokens.

index: int

The index of this particular choice in the list of possible choices.

logprobs: Dict[str, float] | None = None

Optional dictionary containing token log probabilities.

message: Message

The message object containing the role and content of the response.

class LLMChoiceStreaming

Bases: object

Represents a streaming choice or partial response generated by the LLM.

Attributes:

index: The index of this particular choice in the list of possible choices.

delta: The partial update (ChatDelta) for this choice.

finish_reason: Optional reason for why the generation stopped, may be None during streaming.

logprobs: Optional dictionary containing token log probabilities.

__init__(index, delta, finish_reason=None, logprobs=None)
Parameters:

  • index (int)

  • delta (ChatDelta)

  • finish_reason (str | None)

  • logprobs (Dict[str, float] | None)

Return type:

None

delta: ChatDelta

The partial update (ChatDelta) for this choice.

finish_reason: str | None = None

Optional reason for why the generation stopped, may be None during streaming.

index: int

The index of this particular choice in the list of possible choices.

logprobs: Dict[str, float] | None = None

Optional dictionary containing token log probabilities.

class LLMResult

Bases: BaseLLMResult

Represents the complete result from an LLM operation.

Attributes:

id: The unique identifier for this LLM operation.

object: The type of object returned (typically "chat.completion").

created: The timestamp when this result was created.

model: The name or identifier of the model used for generating the result.

choices: A list of possible choices generated by the LLM.

usage: The token usage statistics for this operation.

system_fingerprint: An optional system fingerprint for tracking the model used.

__init__(id, object, created, model, choices, usage, system_fingerprint=None)
Parameters:

  • id (str)

  • object (str)

  • created (int)

  • model (str)

  • choices (List[LLMChoice])

  • usage (LLMUsage)

  • system_fingerprint (str | None)

Return type:

None

choices: List[LLMChoice]

A list of possible choices generated by the LLM.

system_fingerprint: str | None = None

An optional system fingerprint for tracking the model used.

usage: LLMUsage

The token usage statistics for this operation.

class LLMResultStreaming

Bases: BaseLLMResult

Represents a streaming result from an LLM operation.

Attributes:

id: The unique identifier for this LLM operation.

object: The type of object returned (typically "chat.completion.chunk").

created: The timestamp when this result was created.

model: The name or identifier of the model used.

choices: A list of streaming choices generated by the LLM.

usage: optional token usage statistics for this operation.

system_fingerprint: An optional system fingerprint for tracking the model used.

__init__(id, object, created, model, choices, usage=None, system_fingerprint=None)
Parameters:

  • id (str)

  • object (str)

  • created (int)

  • model (str)

  • choices (List[LLMChoiceStreaming])

  • usage (LLMUsage | None)

  • system_fingerprint (str | None)

Return type:

None

choices: List[LLMChoiceStreaming]

A list of streaming choices generated by the LLM.

system_fingerprint: str | None = None

An optional system fingerprint for tracking the model used.

usage: LLMUsage | None = None

optional token usage statistics for this operation.

class LLMUsage

Bases: object

Represents the token usage statistics for an LLM (Large Language Model) operation.

__init__(completion_tokens, prompt_tokens, total_tokens)
Parameters:

  • completion_tokens (int)

  • prompt_tokens (int)

  • total_tokens (int)

Return type:

None

completion_tokens: int

The number of tokens generated by the model in the response.

prompt_tokens: int

The number of tokens in the input prompt.

total_tokens: int

The total number of tokens used, including both prompt and completion tokens.

class ModuleResults

Bases: BaseModuleResults

Represents the results of various modules used in processing an orchestration request.

Attributes:

templating: A list of messages that define the conversation's context or template.

llm: The result from the LLM operation.

input_filtering: The result of any input filtering, if applicable.

output_filtering: The result of any output filtering, if applicable.

input_masking: The result of input masking, if applicable.

output_unmasking: The result of output unmasking, if applicable.

__init__(input_filtering=None, output_filtering=None, input_masking=None, grounding=None, input_translation=None, output_translation=None, llm=None, templating=None, output_unmasking=None)
Parameters:
Return type:

None

llm: LLMResult | None = None

The result from the LLM operation.

output_unmasking: List[LLMChoice] | None = None

The result of output unmasking, if applicable.

templating: List[Message] | None = None

A list of messages that define the conversation's context or template.

class ModuleResultsStreaming

Bases: BaseModuleResults

Represents the streaming results of various modules used in processing an orchestration request.

Attributes:

llm: The streaming result from the LLM operation.

templating: A list of chat deltas that define the conversation's context or template.

input_filtering: The result of any input filtering, if applicable.

output_filtering: The result of any output filtering, if applicable.

input_masking: The result of input masking, if applicable.

output_unmasking: The result of output unmasking for streaming responses.

__init__(input_filtering=None, output_filtering=None, input_masking=None, grounding=None, input_translation=None, output_translation=None, llm=None, templating=None, output_unmasking=None)
Parameters:
Return type:

None

llm: LLMResultStreaming | None = None

The streaming result from the LLM operation.

output_unmasking: List[LLMChoiceStreaming] | None = None

The result of output unmasking for streaming responses.

templating: List[ChatDelta] | None = None

A list of chat deltas that define the conversation's context or template.

class OrchestrationResponse

Bases: object

Represents the complete response from an orchestration process.

Attributes:

request_id: The unique identifier for the request being processed.

module_results: The results from the various modules involved in processing the request.

orchestration_result: The final result from the orchestration, typically mirroring the LLM result.

__init__(request_id, module_results, orchestration_result)
Parameters:
Return type:

None

property content: str

Gets the content of the first choice in the orchestration result.

Raises:

ValueError -- If there are no choices available in the orchestration result.

Returns:

The content of the first choice.

Return type:

str

module_results: ModuleResults

The results from the various modules involved in processing the request.

orchestration_result: LLMResult

The final result from the orchestration, typically mirroring the LLM result.

request_id: str

The unique identifier for the request being processed.

class OrchestrationResponseStreaming

Bases: object

Represents the streaming response from an orchestration process.

Attributes:

request_id: The unique identifier for the request being processed.

module_results: The streaming results from the various modules involved in processing the request.

orchestration_result: The streaming result from the orchestration.

__init__(request_id, module_results, orchestration_result)
Parameters:
Return type:

None

module_results: ModuleResultsStreaming

The streaming results from the various modules involved in processing the request.

orchestration_result: LLMResultStreaming

The streaming result from the orchestration.

request_id: str

The unique identifier for the request being processed.

class OrchestrationResponseWithRetries

Bases: OrchestrationResponse

Extended OrchestrationResponse that includes retry count information.

This is returned when using retry-enabled methods like run_with_retries().

Attributes:

retries: Number of retry attempts that were made to successfully complete this request.

__init__(request_id, module_results, orchestration_result, retries=0)
Parameters:
Return type:

None

retries: int = 0

Number of retry attempts that were made to successfully complete this request.

class ToolCallChunk

Bases: object

Represents a chunk of a tool call in a streaming chat response.

__init__(index, id=None, type=None, function=None)
Parameters:

  • index (int)

  • id (str | None)

  • type (str | None)

  • function (FunctionCall | None)

Return type:

None

function: FunctionCall | None = None

The function call details associated with this tool call chunk.

id: str | None = None

The unique identifier for the tool call.

index: int

The index of this tool call chunk in the sequence of chunks.

type: str | None = None

The type of tool call, e.g., 'function' .

gen_ai_hub.orchestration.models.response_format module

class ResponseFormatFactory

Bases: object

Factory class that maps response format input to classes that can handle to_dict conversion.

static create_response_format_object(response_format)

Creates a response format object based on the provided response format.

Parameters:

response_format (Union[ResponseFormatType, ResponseFormatJsonSchema]) -- The response format input.

Returns:

An instance of the corresponding response format class.

Return type:

Optional[JSONSerializable]

class ResponseFormatJsonObject

Bases: JSONSerializable

Response format JSON Object that the model output should adhere to.

to_dict()

Converts the ResponseFormatJsonObject instance to a dictionary.

Returns:

A dictionary representation of the ResponseFormatJsonObject.

Return type:

dict

class ResponseFormatJsonSchema

Bases: JSONSerializable

Response format JSON Schema that the model output should adhere to.

__init__(name, schema=None, description=None, strict=False)

Initializes a ResponseFormatJsonSchema instance.

Parameters:

  • name (str) -- the name of the response format.

  • schema (object, optional) -- the schema for the response format described as a JSON Schema object, defaults to None

  • description (Optional[str], optional) -- A description of what the response format is for, defaults to None

  • strict (bool, optional) -- Whether to enable strict schema adherence when generating the output, defaults to False

to_dict()

Converts the ResponseFormatJsonSchema instance to a dictionary.

Returns:

A dictionary representation of the ResponseFormatJsonSchema.

Return type:

dict

class ResponseFormatText

Bases: JSONSerializable

Response format that the model output should adhere to.

to_dict()

Converts the ResponseFormatText instance to a dictionary.

Returns:

A dictionary representation of the ResponseFormatText.

Return type:

dict

class ResponseFormatType

Bases: str, Enum

Enumerates the supported response format.

Response format that the model output should adhere to. This is the same as the OpenAI definition.

Values:

TEXT: Response format as text JSON_OBJECT: Response format as json object JSON_SCHEMA: Response format as defined json schema

__new__(value)
JSON_OBJECT = 'json_object'
JSON_SCHEMA = 'json_schema'
TEXT = 'text'
class Validator

Bases: object

A utility class for validating response format names.

This class provides methods to validate the names of response formats to ensure they adhere to specified patterns and length constraints.

static validate_name(name)

Validates the name of the response format.

Parameters:

name (str) -- The name to validate.

Raises:

ValueError -- If the name does not match the required pattern or exceeds the maximum length.

Returns:

The validated name.

Return type:

str

gen_ai_hub.orchestration.models.sap_data_privacy_integration module

class MaskingMethod

Bases: str, Enum

Enumerates the supported masking methods.

This enum defines the two main methods for masking sensitive information: anonymization and pseudonymization. Anonymization irreversibly removes sensitive data, while pseudonymization allows the original data to be recovered.

Values:

ANONYMIZATION: Irreversibly replaces sensitive data with placeholders (e.g., MASKED_ENTITY).

PSEUDONYMIZATION: Replaces sensitive data with reversible placeholders (e.g., MASKED_ENTITY_ID).

__new__(value)
ANONYMIZATION = 'anonymization'

Irreversibly replaces sensitive data with placeholders (e.g., MASKED_ENTITY).

PSEUDONYMIZATION = 'pseudonymization'

Replaces sensitive data with reversible placeholders (e.g., MASKED_ENTITY_ID).

class ProfileEntity

Bases: str, Enum

Enumerates the entity categories that can be masked by the SAP Data Privacy Integration service.

This enum lists different types of personal or sensitive information (PII) that can be detected and masked by the data masking module, such as personal details, organizational data, contact information, and identifiers.

Values:

PERSON: Represents personal names.

ORG: Represents organizational names.

UNIVERSITY: Represents educational institutions.

LOCATION: Represents geographical locations.

EMAIL: Represents email addresses.

PHONE: Represents phone numbers.

ADDRESS: Represents physical addresses.

SAP_IDS_INTERNAL: Represents internal SAP identifiers.

SAP_IDS_PUBLIC: Represents public SAP identifiers.

URL: Represents URLs.

USERNAME_PASSWORD: Represents usernames and passwords.

NATIONAL_ID: Represents national identification numbers.

IBAN: Represents International Bank Account Numbers.

SSN: Represents Social Security Numbers.

CREDIT_CARD_NUMBER: Represents credit card numbers.

PASSPORT: Represents passport numbers.

DRIVING_LICENSE: Represents driving license numbers.

NATIONALITY: Represents nationality information.

RELIGIOUS_GROUP: Represents religious group affiliation.

POLITICAL_GROUP: Represents political group affiliation.

PRONOUNS_GENDER: Represents pronouns and gender identity.

GENDER: Represents gender information.

SEXUAL_ORIENTATION: Represents sexual orientation.

TRADE_UNION: Represents trade union membership.

SENSITIVE_DATA: Represents any other sensitive information.

__new__(value)
ADDRESS = 'profile-address'

Represents physical addresses.

CREDIT_CARD_NUMBER = 'profile-credit-card-number'

Represents credit card numbers.

DRIVING_LICENSE = 'profile-driverlicense'

Represents driving license numbers.

EMAIL = 'profile-email'

Represents email addresses.

GENDER = 'profile-gender'

Represents gender information.

IBAN = 'profile-iban'

Represents International Bank Account Numbers.

LOCATION = 'profile-location'

Represents geographical locations.

NATIONALITY = 'profile-nationality'

Represents nationality information.

NATIONAL_ID = 'profile-nationalid'

Represents national identification numbers.

ORG = 'profile-org'

Represents organizational names.

PASSPORT = 'profile-passport'

Represents passport numbers.

PERSON = 'profile-person'

Represents personal names.

PHONE = 'profile-phone'

Represents phone numbers.

POLITICAL_GROUP = 'profile-political-group'

Represents political group affiliation.

PRONOUNS_GENDER = 'profile-pronouns-gender'

Represents pronouns and gender identity.

RELIGIOUS_GROUP = 'profile-religious-group'

Represents religious group affiliation.

SAP_IDS_INTERNAL = 'profile-sapids-internal'

Represents internal SAP identifiers.

SAP_IDS_PUBLIC = 'profile-sapids-public'

Represents public SAP identifiers.

SENSITIVE_DATA = 'profile-sensitive-data'

Represents any other sensitive information.

SEXUAL_ORIENTATION = 'profile-sexual-orientation'

Represents sexual orientation.

SSN = 'profile-ssn'

Represents Social Security Numbers.

TRADE_UNION = 'profile-trade-union'

Represents trade union membership.

UNIVERSITY = 'profile-university'

Represents educational institutions.

URL = 'profile-url'

Represents URLs.

USERNAME_PASSWORD = 'profile-username-password'

Represents usernames and passwords.

class SAPDataPrivacyIntegration

Bases: DataMaskingProvider

SAP Data Privacy Integration provider for data masking.

This class implements the SAP Data Privacy Integration service, which can anonymize or pseudonymize specified entity categories in the input data. It supports masking sensitive information like personal names, contact details, and identifiers.

__init__(method, entities, allowlist=None, mask_grounding_input=False)

Initializes the SAPDataPrivacyIntegration data masking provider.

Parameters:

  • method (MaskingMethod) -- The method of masking to apply

  • entities (List[ProfileEntity]) -- A list of entity categories to be masked

  • allowlist (List[str], optional) -- A list of strings that should not be masked, defaults to None

  • mask_grounding_input (bool, optional) -- A flag indicating whether to mask input to the grounding module, defaults to False

to_dict()

Converts the SAPDataPrivacyIntegration instance to a dictionary representation.

Returns:

Dictionary representation of the SAPDataPrivacyIntegration instance.

Return type:

dict

gen_ai_hub.orchestration.models.template module

class Template

Bases: JSONSerializable

Represents a configurable template for generating prompts or conversations.

__init__(messages, defaults=None, tools=None, response_format=None)

Initializes a Template instance.

Parameters:

  • messages (List[Message]) -- list of prompt messages that form the template._

  • defaults (Optional[List[TemplateValue]], optional) -- list of default values for template variables, defaults to None

  • tools (Optional[List[Union[dict, ChatCompletionTool]]], optional) -- list of tool definitions, defaults to None

  • response_format (Optional[Union[ ResponseFormatType.TEXT, ResponseFormatType.JSON_OBJECT, ResponseFormatJsonSchema ]], optional) -- response format that the model output should adhere to, defaults to None

to_dict()
Converts the Template instance to a dictionary representation.

Serializes the template to a dictionary, converting tools as needed.

Raises:

ValueError -- If an invalid tool is encountered in the tools list.

Returns:

A dictionary representation of the Template instance.

Return type:

Dict[str, Any]

class TemplateValue

Bases: NamedTuple

Represents a named value for use in template substitution.

This class pairs a name with a corresponding value, which can be a string, integer, or float. It's designed to be used in template rendering processes where named placeholders are replaced with specific values.

static __new__(_cls, name, value)

Create new instance of TemplateValue(name, value)

Parameters:

  • name (str)

  • value (str | int | float)

name: str

The identifier for this template value.

value: str | int | float

The actual value to be used in substitution.

gen_ai_hub.orchestration.models.template_ref module

class TemplateRef

Bases: JSONSerializable

Represents a prompt template reference for generating prompts or conversations.

This is a factory class for creating a reference to a prompt template. It is used to reference a template by id, or the tuple: scenario, name, version

classmethod from_id(prompt_template_id)

Creates a TemplateRef instance from a prompt template ID.

Parameters:

prompt_template_id (str) -- The ID of the prompt template.

Returns:

A TemplateRef instance with the specified ID.

Return type:

TemplateRef

classmethod from_tuple(scenario, name, version)

Creates a TemplateRef instance from a scenario, name, and version.

Parameters:

  • scenario (str) -- The scenario of the prompt template.

  • name (str) -- The name of the prompt template.

  • version (str) -- The version of the prompt template.

Returns:

A TemplateRef instance with the specified scenario, name, and version.

Return type:

TemplateRef

__init__(**kwargs)

Initializes a TemplateRef instance with dynamic attributes.

to_dict()

Converts the TemplateRef instance to a dictionary representation.

Returns:

A dictionary representation of the TemplateRef instance.

Return type:

dict

gen_ai_hub.orchestration.models.tools module

class ChatCompletionTool

Bases: JSONSerializable

Base class for all chat completion tools.

__init__(type_)

Initializes a ChatCompletionTool instance.

Parameters:

  • type (str) -- The type of the tool.

  • type_ (str)

to_dict()

Converts the ChatCompletionTool instance to a dictionary representation.

Returns:

A dictionary representation of the ChatCompletionTool instance.

Return type:

Dict[str, Any]

class FunctionTool

Bases: ChatCompletionTool

Represents a function tool for OpenAI-like function calling.

static from_function(func, *, description=None, strict=False)

Create a FunctionTool from a Python function.

Parameters:

  • func (Callable) -- The Python function to convert.

  • description (Optional[str], optional) -- The description of the function, defaults to None

  • strict (bool, optional) -- Whether to enforce strict parameter checking, defaults to False

Raises:

TypeError -- If any parameter is missing a type hint.

Returns:

A FunctionTool instance.

Return type:

FunctionTool

__init__(name, parameters, strict=False, description=None, function=None)

Initializes a FunctionTool instance.

Parameters:

  • name (str) -- The name of the function.

  • parameters (dict) -- The parameters schema for the function.

  • strict (bool, optional) -- Whether to enforce strict parameter checking, defaults to False

  • description (Optional[str], optional) -- The description of the function, defaults to None

  • function (Optional[Callable], optional) -- The actual callable function, defaults to None

async aexecute(**kwargs)

Asynchronously execute the function with the provided arguments.

Raises:

ValueError -- If the function is not set or if unexpected arguments are provided in strict mode.

Returns:

The result of the function execution.

Return type:

Any

Parameters:

kwargs (Any)

execute(**kwargs)

Execute the function with the provided arguments.

Raises:

ValueError -- If the function is not set or if unexpected arguments are provided in strict mode.

Returns:

The result of the function execution.

Return type:

Any

Parameters:

kwargs (Any)

to_dict()

Converts the FunctionTool instance to a dictionary representation.

Returns:

A dictionary representation of the FunctionTool instance.

Return type:

Dict[str, Any]

function_tool(func=None, *, description=None, strict=False)

Create a decorator that converts a function into a FunctionTool.

Usage:

@function_tool

def my_func(...): ...

@function_tool()

def my_func(...): ...

Parameters:

  • func (Optional[Callable], optional) -- The function to convert, defaults to None

  • description (Optional[str], optional) -- The description of the function, defaults to None

  • strict (bool, optional) -- Whether to enforce strict parameter checking, defaults to False

Returns:

A FunctionTool instance or a decorator function.

Return type:

Callable[[Callable], FunctionTool] | FunctionTool

python_type_to_json_type(py_type)

Converts a Python type to a JSON Schema type.

Parameters:

py_type (type) -- The Python type to convert.

Returns:

A dictionary representing the JSON Schema type.

Return type:

Dict[str, Any]