gen_ai_hub.proxy.core package

get_proxy_client(proxy_version=None, catalog=None, **kwargs)

Get a proxy client for the given proxy version.

Parameters:

  • proxy_version (str | None, optional) -- The version of the proxy client to retrieve. If not provided, the function will attempt to retrieve the version using the get_proxy_version function.

  • catalog (Optional[ProxyClients], optional) -- The catalog from which to retrieve the proxy client. If not provided, the function will default to the proxy_clients catalog

  • kwargs (dict) -- Arbitrary keyword arguments that will be passed to the constructor of the proxy client class.

Returns:

An instance of the proxy client.

Return type:

BaseProxyClient

get_proxy_version(catalog=None)

Get the current proxy version. The version is selected in the following order:

  • thread-local overwrite (set with proxy_version_context)

  • global overwrite (set with set_proxy_version)

  • environment variable (LLM_PROXY_VERSION)

  • first registered proxy version

Parameters:

catalog (Optional[ProxyClients], optional) -- The catalog from which to get the proxy version. If none is provided,

Raises:

ValueError -- If no proxy version is set.

Returns:

The current proxy version.

Return type:

str

proxy_version_context(proxy_version, catalog=None)

Context manager to set a thread-local proxy version.

Parameters:

  • proxy_version (str) -- The proxy version to set.

  • catalog (Optional[ProxyClients], optional) -- The catalog for which the proxy version is to be set. If none is provided, the proxy version is set for the default proxy_clients catalog.

Raises:

ValueError -- If proxy_version is not a string.

Return type:

None

set_proxy_version(proxy_version, catalog=None)

Set the global proxy version.

Parameters:

  • proxy_version (str) -- The proxy version to set.

  • catalog (Optional[ProxyClients], optional) -- The catalog for which the proxy version is to be set. If none is provided, the proxy version is set for the default proxy_clients catalog.

Raises:

ValueError -- If proxy_version is not a string.

Return type:

None

Submodules

gen_ai_hub.proxy.core.base module

class BaseDeployment

Bases: BaseModel, ABC

Abstract base class for all deployment types.

Parameters:

  • BaseModel (pydantic.BaseModel) -- the base model class from Pydantic.

  • ABC (abc.ABC) -- the abstract base class module.

Returns:

the abstract base class for deployments.

Return type:

BaseDeployment

classmethod get_main_model_identification_kwargs()
Return type:

str

abstractmethod classmethod get_model_identification_kwargs()
Return type:

Tuple[str]

abstractmethod additional_request_body_kwargs()
Return type:

Dict[str, Any]

model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

abstract property prediction_url: Tuple[str]
class BaseProxyClient

Bases: ABC, BaseModel

Abstract base class for all proxy clients.

classmethod refresh_instance_cache()

Refresh the cache of instances.

abstractmethod select_deployment(**kwargs)
Return type:

BaseDeployment

abstract property deployment_class: Type[BaseDeployment]
abstract property deployments: Dict[str, Any]
model_config: ClassVar[ConfigDict] = {'protected_namespaces': ()}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

abstract property request_header: Dict[str, Any]
class CombinedMeta

Bases: InstanceCacheMeta, ModelMetaclass

class InstanceCacheMeta

Bases: type

Metaclass that caches instances based on their initialization arguments.

Parameters:

type (type) -- the metaclass type.

Returns:

the metaclass that caches instances.

Return type:

InstanceCacheMeta

clear_cache()

Clear the instance cache.

gen_ai_hub.proxy.core.proxy_clients module

class ProxyClients

Bases: object

Catalog for proxy client classes.

__init__()

Initializes a ProxyClients instance.

Return type:

None

get_proxy_cls(proxy_version=None)

Get the proxy client class for the given version.

Parameters:

proxy_version (str | None, optional) -- The proxy version.

Returns:

The proxy client class.

Return type:

type[BaseProxyClient]

get_proxy_cls_name(proxy_client_cls)

Get the name of the proxy client class.

Parameters:

proxy_client_cls (type[BaseProxyClient] | BaseProxyClient) -- The proxy client class or instance.

Raises:

  • ValueError -- If the provided class is not a subclass of BaseProxyClient.

  • KeyError -- If the class is not registered.

Returns:

The name of the proxy client class.

Return type:

str

register(name)

Decorator to register a proxy client class.

Parameters:

name (str) -- The name to register the proxy client class under.

Raises:

ValueError -- If the provided class is not a subclass of BaseProxyClient.

Returns:

A wrapper function.

Return type:

Callable

get_proxy_client(proxy_version=None, catalog=None, **kwargs)

Get a proxy client for the given proxy version.

Parameters:

  • proxy_version (str | None, optional) -- The version of the proxy client to retrieve. If not provided, the function will attempt to retrieve the version using the get_proxy_version function.

  • catalog (Optional[ProxyClients], optional) -- The catalog from which to retrieve the proxy client. If not provided, the function will default to the proxy_clients catalog

  • kwargs (dict) -- Arbitrary keyword arguments that will be passed to the constructor of the proxy client class.

Returns:

An instance of the proxy client.

Return type:

BaseProxyClient

get_proxy_version(catalog=None)

Get the current proxy version. The version is selected in the following order:

  • thread-local overwrite (set with proxy_version_context)

  • global overwrite (set with set_proxy_version)

  • environment variable (LLM_PROXY_VERSION)

  • first registered proxy version

Parameters:

catalog (Optional[ProxyClients], optional) -- The catalog from which to get the proxy version. If none is provided,

Raises:

ValueError -- If no proxy version is set.

Returns:

The current proxy version.

Return type:

str

proxy_version_context(proxy_version, catalog=None)

Context manager to set a thread-local proxy version.

Parameters:

  • proxy_version (str) -- The proxy version to set.

  • catalog (Optional[ProxyClients], optional) -- The catalog for which the proxy version is to be set. If none is provided, the proxy version is set for the default proxy_clients catalog.

Raises:

ValueError -- If proxy_version is not a string.

Return type:

None

set_proxy_version(proxy_version, catalog=None)

Set the global proxy version.

Parameters:

  • proxy_version (str) -- The proxy version to set.

  • catalog (Optional[ProxyClients], optional) -- The catalog for which the proxy version is to be set. If none is provided, the proxy version is set for the default proxy_clients catalog.

Raises:

ValueError -- If proxy_version is not a string.

Return type:

None

gen_ai_hub.proxy.core.utils module

class NotGiven

Bases: object

class Omit

Bases: object

class PredictionURLs

Bases: object

Generate prediction URLs with dynamic suffixes for different models.

__init__(suffixes=None)

Initialize the PredictionURLs with optional suffixes.

Parameters:

suffixes (Optional[Dict[str, str]], optional) -- A dictionary of model names and their corresponding URL suffixes.

Return type:

None

register(suffixes)

Register new model suffixes.

Parameters:

suffixes (Dict[str, str | Omit]) -- A dictionary of model names and their corresponding URL suffixes.

Return type:

None

if_set(value, alternative=<gen_ai_hub.proxy.core.utils.NotGiven object>)

Check if a value is set (not NotGiven and not None), otherwise return an alternative.

Parameters:

  • value (any) -- the value to check.

  • alternative (any) -- the alternative value to return if the original value is not set.

Returns:

The original value if set, otherwise the alternative.

Return type:

any

if_str_set(value, alternative='')

Check if a string value is set (not empty), otherwise return an alternative.

Parameters:

  • value (str) -- the string value to check.

  • alternative (str, optional) -- the alternative string to return if the original value is empty.

Returns:

The original string if not empty, otherwise the alternative.

Return type:

str

kwargs_if_set(**kwargs)

Filter keyword arguments to include only those that are set (not NotGiven and not None).

Returns:

A dictionary of keyword arguments that are set.

Return type:

Dict[str, any]

lru_cache_extended(timeout=None, maxsize=None, typed=False, first_arg_self=False)

Decorator to add LRU caching with optional timeout to methods. Handles 'self' as a weak reference for instance methods if required.

Parameters:

  • timeout (Optional[int], optional) -- time in seconds after which the cache will be refreshed. If None, never expires.

  • maxsize (Optional[int], optional) -- maximum size of the cache.

  • typed (bool, optional) -- if True, arguments of different types will be cached separately.

  • first_arg_self (bool, optional) -- if True, treats the first argument as 'self' and uses its id for caching.

Returns:

Decorated method with cache and optional timeout.

Return type:

Callable

warn_once(msg, category=None)

Issue a warning only once for a given message.

Parameters:

  • msg (str) -- the warning message.

  • category (Optional[Warning], optional) -- the warning category.