gen_ai_hub.proxy.native.openai package

class GlobalClient

Bases: object

A global client to manage OpenAI clients based on proxy version.

__init__()

Return type:: None

property client

Get the OpenAI client based on the current proxy version.

Returns:: The OpenAI client instance.
Return type:: OpenAI

Submodules

gen_ai_hub.proxy.native.openai.clients module

class AsyncChat

Bases: AsyncChat

A class that handles asynchronous chat completions, extending from the class 'AsyncChat_'.

__init__(client)

Initializes the AsyncChat class with the provided OpenAI client.

Parameters:: client (OpenAI) -- The OpenAI client to be used for chat completions.
Return type:: None

class AsyncChatCompletions

Bases: AsyncCompletions

The AsyncChatCompletions class is a derived class which extends AsyncChatCompletions_. This class is used to handle asynchronous chat completion requests. It provides methods to create and manage chat completions in an asynchronous manner.

static supports_temperature(model_name)

Checks if the given model supports the temperature parameter. Reasoning models do not support temperature e.g., o1[-mini], o3[-mini], cohere--command-a-reasoning

Parameters:: model_name (str) -- the name of the model to check.
Returns:: True if the model supports temperature, False otherwise
Return type:: bool

async create(*, messages, model=<gen_ai_hub.proxy.core.utils.NotGiven object>, deployment_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, model_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, **kwargs)

Asynchronously creates a new chat completion.

Parameters:

messages (List[ChatCompletionMessageParam]) -- the list of chat completion message parameters.
model (str | None | NotGiven, optional) -- the model to be used, defaults to NOT_GIVEN
deployment_id (str | None | NotGiven, optional) -- the deployment id, defaults to NOT_GIVEN
model_name (str | None | NotGiven, optional) -- the model name, defaults to NOT_GIVEN
config_id (str | None | NotGiven, optional) -- the configuration id, defaults to NOT_GIVEN
config_name (str | None | NotGiven, optional) -- the configuration name, defaults to NOT_GIVEN

Returns:

the created chat completion.

Return type:

ChatCompletion

async parse(*, messages, model=<gen_ai_hub.proxy.core.utils.NotGiven object>, deployment_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, model_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, response_format=<gen_ai_hub.proxy.core.utils.NotGiven object>, **kwargs)

Asynchronously parses chat completions using the provided parameters and returns a ParsedChatCompletion object. This method provides richer integrations with Python specific types by converting pydantic models into JSON schemas and parsing the response content back into the given model.

Parameters:

messages (Iterable[ChatCompletionMessageParam]) -- the list of chat completion message parameters.
model (str | None | NotGiven, optional) -- the model to use for chat completion, defaults to NOT_GIVEN
deployment_id (str | None | NotGiven, optional) -- the deployment ID to use for chat completion, defaults to NOT_GIVEN
model_name (str | None | NotGiven, optional) -- the model name to use for chat completion, defaults to NOT_GIVEN
config_id (str | None | NotGiven, optional) -- the configuration ID to use for chat completion, defaults to NOT_GIVEN
config_name (str | None | NotGiven, optional) -- the configuration name to use for chat completion, defaults to NOT_GIVEN
response_format (type[ResponseFormatT] | NotGiven, optional) -- the response format type for structured output, defaults to NOT_GIVEN

Returns:

the parsed chat completion with the structured response.

Return type:

ParsedChatCompletion[ResponseFormatT]

class AsyncCompletions

Bases: AsyncCompletions

AsyncCompletions is a subclass of AsyncCompletions_. It provides a way to create a completion given a prompt and: certain other configurations in asynchronous way. It extends from the base class Completions_ and overrides the create method to cater to the specific requirements.

async create(*, prompt, model=<gen_ai_hub.proxy.core.utils.NotGiven object>, deployment_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, model_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, **kwargs)

Asynchronously creates a completion or a stream of completions based on the given prompt and other parameters.

Parameters:

prompt (Union[str, List[str], List[int], List[List[int]], None]) -- the input prompt(s) for the completion.
model (str | None | NotGiven, optional) -- the model to be used for the completion, defaults to NOT_GIVEN
deployment_id (str | None | NotGiven, optional) -- the deployment id, defaults to NOT_GIVEN
model_name (str | None | NotGiven, optional) -- the model name, defaults to NOT_GIVEN
config_id (str | None | NotGiven, optional) -- the configuration id, defaults to NOT_GIVEN
config_name (str | None | NotGiven, optional) -- the configuration name, defaults to NOT_GIVEN

Returns:

the completion or stream of completions created based on the provided prompt.

Return type:

Completion | Stream[Completion]

class AsyncEmbeddings

Bases: AsyncEmbeddings

The AsyncEmbeddings class is a subclass of AsyncEmbeddings_. This class is used for creating embeddings asynchronously. It provides an interface for fetching embeddings of a given input from a selected deployment on a proxy client.

async create(*, input, model=<gen_ai_hub.proxy.core.utils.NotGiven object>, deployment_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, model_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, **kwargs)

Asynchronously creates embeddings for the given input using a specific model.

Parameters:

input (Union[str, List[str], List[int], List[List[int]], None]) -- the input data for which embeddings are to be created.
model (str | None | NotGiven, optional) -- the model to use for creating embeddings, defaults to NOT_GIVEN
deployment_id (str | None | NotGiven, optional) -- the ID of the deployment to use, defaults to NOT_GIVEN
model_name (str | None | NotGiven, optional) -- the name of the model to use, defaults to NOT_GIVEN
config_id (str | None | NotGiven, optional) -- the ID of the config to use, defaults to NOT_GIVEN
config_name (str | None | NotGiven, optional) -- the name of the config to use, defaults to NOT_GIVEN

Returns:

the created embeddings.

Return type:

Embedding

class AsyncOpenAI

Bases: AsyncOpenAI

An async version of the OpenAI API client.

This class is used to interact with the OpenAI API asynchronously. It supports various operations like creating completions, generating chat messages, and getting embeddings.

Attributes:

proxy_client (BaseProxyClient): A proxy client to make API requests. If not provided, a default one will be created.

api_version (str, optional): The version of the OpenAI API to use. Default is defined by DEFAULT_API_VERSION.

completions (AsyncCompletions): A client for interacting with the OpenAI API's completions.

chat (AsyncChat): A client for interacting with the OpenAI API's chat.

edits (None): Placeholder for future support of "edits" operations.

embeddings (AsyncEmbeddings): A client for interacting with the OpenAI API's embeddings.

files (None): Placeholder for future support of "files" operations.

images (None): Placeholder for future support of "images" operations.

audio (None): Placeholder for future support of "audio" operations.

moderations (None): Placeholder for future support of "moderations" operations.

models (None): Placeholder for future support of "models" operations.

fine_tuning (None): Placeholder for future support of "fine_tuning" operations.

fine_tunes (None): Placeholder for future support of "fine_tunes" operations. beta (None): Placeholder for future support of "beta" operations. with_raw_response (AsyncOpenAIWithRawResponse): A client that returns raw API responses.

__init__(*, proxy_client=None, api_version='2025-03-01-preview', **kwargs)

Initializes the AsyncOpenAI client with the provided parameters.

Parameters:

proxy_client (Optional[BaseProxyClient], optional) -- An instance of a Proxy Client. Defaults to None.
api_version (Optional[str], optional) -- API version used for OpenAI API calls. Defaults to DEFAULT_API_VERSION.

Return type:

None

request(cast_to, options, *args, **kwargs)

Overrides the request method to include additional request body kwargs from the current deployment.

Parameters:

cast_to (any) -- the type to cast the response to.
options (any) -- the request options.

Returns:

the response from the request.

Return type:

CoroutineType[Any, Any, ResponseT@request]

property default_headers: dict[str, str | Omit]

class AsyncOpenAIWithRawResponse

Bases: object

A class that provides an asynchronous interface to the OpenAI API, returning raw responses.

This class wraps the core functionality of OpenAI's API, offering access to completions, chat capabilities, and embeddings. It is designed to work with OpenAI's asynchronous client, allowing for concurrent requests to the API.

Note: The properties 'edits', 'files', 'images', 'audio', 'moderations', 'models', 'fine_tuning', 'fine_tunes' and 'beta' are placeholders and currently do not provide any functionality.

Attributes:

completions: An instance of resources.AsyncCompletionsWithRawResponse for managing completions with the API.

chat: An instance of resources.AsyncChatWithRawResponse for managing chat with the API.

embeddings: An instance of resources.AsyncEmbeddingsWithRawResponse for managing embeddings with the API.

edits: Currently a placeholder with no functionality.

files: Currently a placeholder with no functionality.

images: Currently a placeholder with no functionality.

audio: Currently a placeholder with no functionality.

moderations: Currently a placeholder with no functionality.

models: Currently a placeholder with no functionality.

fine_tuning: Currently a placeholder with no functionality.

fine_tunes: Currently a placeholder with no functionality.

beta: Currently a placeholder with no functionality.

__init__(client)

Initializes the AsyncOpenAIWithRawResponse class with the provided AsyncOpenAI client.

Parameters:: client (AsyncOpenAI) -- An instance of AsyncOpenAI client.
Return type:: None

class Chat

Bases: Chat

A class that handles chat completions, extending from the class 'Chat_'.

__init__(client)

Initializes the Chat class with the provided OpenAI client.

Parameters:: client (OpenAI) -- The OpenAI client to be used for chat completions.
Return type:: None

class ChatCompletions

Bases: Completions

A class that handles chat completions, extending from the class 'ChatCompletions_'.

static supports_temperature(model_name)

Checks if the given model supports the temperature parameter. Reasoning models do not support temperature e.g., o1[-mini], o3[-mini], 5[-mini, -nano], cohere--command-a-reasoning

Parameters:: model_name (str) -- the name of the model to check.
Returns:: True if the model supports temperature, False otherwise
Return type:: bool

create(*, messages, model=<gen_ai_hub.proxy.core.utils.NotGiven object>, deployment_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, model_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, **kwargs)

Creates a chat completion using the provided parameters.

Parameters:

messages (List[ChatCompletionMessageParam]) -- the list of chat completion message parameters.
model (str | None | NotGiven, optional) -- the model to use for chat completion, defaults to NOT_GIVEN
deployment_id (str | None | NotGiven, optional) -- the deployment ID to use for chat completion, defaults to NOT_GIVEN
model_name (str | None | NotGiven, optional) -- the model name to use for chat completion, defaults to NOT_GIVEN
config_id (str | None | NotGiven, optional) -- the configuration ID to use for chat completion, defaults to NOT_GIVEN
config_name (str | None | NotGiven, optional) -- the configuration name to use for chat completion, defaults to NOT_GIVEN

Returns:

the chat completion created with the provided parameters.

Return type:

ChatCompletion

parse(*, messages, model=<gen_ai_hub.proxy.core.utils.NotGiven object>, deployment_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, model_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, response_format=<gen_ai_hub.proxy.core.utils.NotGiven object>, **kwargs)

Parses chat completions using the provided parameters and returns a ParsedChatCompletion object. This method provides richer integrations with Python specific types by converting pydantic models into JSON schemas and parsing the response content back into the given model.

Parameters:

messages (Iterable[ChatCompletionMessageParam]) -- the list of chat completion message parameters.
model (str | None | NotGiven, optional) -- the model to use for chat completion, defaults to NOT_GIVEN
deployment_id (str | None | NotGiven, optional) -- the deployment ID to use for chat completion, defaults to NOT_GIVEN
model_name (str | None | NotGiven, optional) -- the model name to use for chat completion, defaults to NOT_GIVEN
config_id (str | None | NotGiven, optional) -- the configuration ID to use for chat completion, defaults to NOT_GIVEN
config_name (str | None | NotGiven, optional) -- the configuration name to use for chat completion, defaults to NOT_GIVEN
response_format (type[ResponseFormatT] | NotGiven, optional) -- the response format type for structured output, defaults to NOT_GIVEN

Returns:

the parsed chat completion with the structured response.

Return type:

ParsedChatCompletion[ResponseFormatT]

class Completions

Bases: Completions

The Completions class is a subclass of Completions_. It provides a way to create a completion given a prompt and: certain other configurations. It extends from the base class Completions_ and overrides the create method to cater to the specific requirements.

create(*, prompt, model=<gen_ai_hub.proxy.core.utils.NotGiven object>, deployment_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, model_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, **kwargs)

This method creates a completion based on the provided parameters. It uses a proxy client to select a deployment and then calls the create method of the parent class to generate a completion.

Parameters:

prompt (Union[str, List[str], List[int], List[List[int]], None]) -- the input prompt(s) for the completion.
model (str | None | NotGiven, optional) -- the model to be used for the completion, defaults to NOT_GIVEN
deployment_id (str | None | NotGiven, optional) -- the deployment id, defaults to NOT_GIVEN
model_name (str | None | NotGiven, optional) -- the model name, defaults to NOT_GIVEN
config_id (str | None | NotGiven, optional) -- the configuration id, defaults to NOT_GIVEN
config_name (str | None | NotGiven, optional) -- the configuration name, defaults to NOT_GIVEN

Returns:

the completion or stream of completions created based on the provided prompt.

Return type:

Completion | Stream[Completion]

class Embeddings

Bases: Embeddings

A class that represents the Embeddings. It extends the Embeddings_ class and provides functionality to create embeddings based on the provided input.

create(*, input, model=<gen_ai_hub.proxy.core.utils.NotGiven object>, deployment_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, model_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_id=<gen_ai_hub.proxy.core.utils.NotGiven object>, config_name=<gen_ai_hub.proxy.core.utils.NotGiven object>, **kwargs)

Creates embeddings based on the provided input and model information.

For NVIDIA models, use extra_body to specify additional parameters:: extra_body={'input_type': 'query'|'passage'}

Parameters:

input (Union[str, List[str], List[int], List[List[int]], None]) -- the input data for which embeddings are to be created.
model (str | None | NotGiven)
deployment_id (str | None | NotGiven)
model_name (str | None | NotGiven)
config_id (str | None | NotGiven)
config_name (str | None | NotGiven)

Return type:

Embedding

:param model:the model to use for creating embeddings, defaults to NOT_GIVEN :type model: str | None | NotGiven, optional :param deployment_id: the ID of the deployment to use, defaults to NOT_GIVEN :type deployment_id: str | None | NotGiven, optional :param model_name: the name of the model to use, defaults to NOT_GIVEN :type model_name: str | None | NotGiven, optional :param config_id: the ID of the config to use, defaults to NOT_GIVEN :type config_id: str | None | NotGiven, optional :param config_name: the name of the config to use, defaults to NOT_GIVEN :type config_name: str | None | NotGiven, optional :param kwargs: additional keyword arguments. :type kwargs: dict :raises ValueError: if the deployment cannot be selected or the model name is not provided. :return: the created embeddings. :rtype: Embedding

class OpenAI

Bases: OpenAI

This is a class for the OpenAI API client. It is designed to handle various services provided by OpenAI such as text

completions, chat, embeddings etc.

Attributes:

proxy_client (BaseProxyClient, optional): An instance of a Proxy Client. Defaults to None.

api_version (str, optional): API version used for OpenAI API calls. Defaults to DEFAULT_API_VERSION.

completions (Completions): An instance of the Completions class for text generation.

chat (Chat): An instance of the Chat class for conversation.

edits: Placeholder for future use. Currently set to None.

embeddings (Embeddings): An instance of the Embeddings class for getting text embeddings.

files: Placeholder for future use. Currently set to None.

images: Placeholder for future use. Currently set to None.

audio: Placeholder for future use. Currently set to None.

moderations: Placeholder for future use. Currently set to None.

models: Placeholder for future use. Currently set to None.

fine_tuning: Placeholder for future use. Currently set to None.

fine_tunes: Placeholder for future use. Currently set to None.

beta: Placeholder for future use. Currently set to None.

with_raw_response (OpenAIWithRawResponse): An instance of the OpenAIWithRawResponse class for returning raw responses from the API.

__init__(*, proxy_client=None, api_version='2025-03-01-preview', **kwargs)

Initializes the OpenAI API client with the provided parameters.

Parameters:

proxy_client (Optional[BaseProxyClient], optional) -- An instance of a Proxy Client. Defaults to None.
api_version (Optional[str], optional) -- API version used for OpenAI API calls. Defaults to DEFAULT_API_VERSION.

Return type:

None

request(cast_to, options, *args, **kwargs)

property default_headers: dict[str, str | Omit]

class OpenAIWithRawResponse

Bases: object

This class is a wrapper for the OpenAI API client that provides raw responses. Note: The properties 'edits', 'files', 'images', 'audio', 'moderations', 'models', 'fine_tuning', 'fine_tunes' and 'beta' are placeholders and currently do not provide any functionality.

Attributes:

completions: An instance of CompletionsWithRawResponse class.

chat: An instance of ChatWithRawResponse class.

edits: Not currently used.

embeddings: An instance of EmbeddingsWithRawResponse class if client.embeddings is not None.

files: Not currently used.

images: Not currently used.

audio: Not currently used.

moderations: Not currently used.

models: Not currently used.

fine_tuning: Not currently used.

fine_tunes: Not currently used.

beta: Not currently used.

The class is designed to provide the raw responses from OpenAI's API endpoints. It currently supports completions, chat, and embeddings endpoints.

__init__(client)

Initializes the OpenAIWithRawResponse class with the provided OpenAI client.

Parameters:: client (OpenAI) -- An instance of OpenAI client.
Return type:: None

get_current_deployment()

Get the current deployment from the context variable.

Returns:: The current deployment.
Return type:: Deployment

set_deployment(value)

Context manager to set the current deployment.

Parameters:: value (Deployment) -- The deployment to set as current.