Prompt Registry

The Prompt Registry API allows you to create, manage, and retrieve prompt and orchestration config templates for use in SAP AICore when working with the models of the Generative AI Hub.

In this notebook, we'll walk through the design-time process of creating prompt and config templates using the SDK. (This is also known as imperative Prompt Template creation.) Here, one can create and modify the Prompt Template and Orchestration Config Template. It will be versioned automatically and the versions can be retrieved by an id.

See SAP Help for the difference between imperative and declarative prompt templates.

Prerequisite

Provide the credentials to authenticate the client and establish a connection with the Prompt Registry API.

See options for providing credentials here.

Prompt Template Management

Step 0: Initialize Client

Initialize the client to interact with the Prompt Registry.

from gen_ai_hub.proxy import get_proxy_client
from gen_ai_hub.prompt_registry.client import PromptTemplateClient
proxy_client = get_proxy_client(proxy_version="gen-ai-hub")
prompt_registry_client = PromptTemplateClient(proxy_client=proxy_client)

Step 1: Create Prompt Templates

Define the Prompt Template configuration and post to Prompt Registry.

from gen_ai_hub.prompt_registry.models.prompt_template import PromptTemplateSpec, PromptTemplate

prompt_template_spec = PromptTemplateSpec(template=[PromptTemplate(role='system', content='You are a helpful assistant.')])

template_id = prompt_registry_client.create_prompt_template(
    scenario='MyScenario', name='prompt_template_name', version='1.0.0', prompt_template_spec=prompt_template_spec).id

print(f"Created Prompt Template with ID: {template_id}")

Step 2: Retrieve Prompt Templates

Retrieve the Prompt Template by ID.

response = prompt_registry_client.get_prompt_template_by_id(template_id)
print(response.spec.template)

Step 3: Modify the Prompt Template

We will add an input variable to the existing Prompt Template.

prompt_template_spec = PromptTemplateSpec(template=[PromptTemplate(role='system',
                                                                   content='You are a helpful assistant for {{ ?topic }}.'
                                                                   )
                                                   ]
                                          )
response = prompt_registry_client.create_prompt_template(scenario='MyScenario', name='prompt_template_name', version='1.0.0',
                                                         prompt_template_spec=prompt_template_spec)

input_template_id = response.id
print(response.message)

Step 4: Prompt Template History

Retrieve the history of Prompt Templates by scenario, name and version.

response = prompt_registry_client.get_prompt_template_history(scenario='MyScenario', name='prompt_template_name', version='1.0.0')
print(response.json())

Step 5: Fill Prompt Template

Fill the variables in the Prompt Template.

response = prompt_registry_client.fill_prompt_template_by_id(template_id=input_template_id, input_params={"topic": "chemistry"})
print(response.parsed_prompt)

Orchestration Config Management

Step 0: Initialize Client

Initialize the client to interact with the Prompt Registry.

from gen_ai_hub.proxy import get_proxy_client
from gen_ai_hub.prompt_registry.client import OrchestrationConfigClient
proxy_client = get_proxy_client(proxy_version="gen-ai-hub")
prompt_registry_client = OrchestrationConfigClient(proxy_client=proxy_client)

Step 1: Create Orchestration Config

Define the Orchestration Config configuration and post to Prompt Registry.

from gen_ai_hub.orchestration_v2.models.config import OrchestrationConfig, ModuleConfig
from gen_ai_hub.orchestration_v2.models.llm_model_details import LLMModelDetails
from gen_ai_hub.orchestration_v2.models.message import UserMessage
from gen_ai_hub.orchestration_v2.models.template import Template, PromptTemplatingModuleConfig

config_spec = OrchestrationConfig(
            modules=ModuleConfig(
                prompt_templating=PromptTemplatingModuleConfig(
                    prompt=Template(template=[UserMessage(content="Hello, World!")]),
                    model=LLMModelDetails(name="gpt-4o-mini")
                )
            )
        )

template_id = prompt_registry_client.create_orchestration_config(
    scenario='MyScenario', name='prompt_template_name', version='1.0.0', spec=config_spec).id

print(f"Created Orchestration Config Template with ID: {template_id}")

Step 2: Retrieve Orchestration Config

Retrieve the Orchestration Config by ID.

response = prompt_registry_client.get_orchestration_config_by_id(template_id)
print(response.spec)

Retrieve the Orchestration Configs by scenario, name and version..

response = prompt_registry_client.get_orchestration_configs(scenario='MyScenario', name='prompt_template_name', version='1.0.0')
print(response.resources)

Step 3: Orchestration Config History

Retrieve the history of Orchestration Config by scenario, name and version.

response = prompt_registry_client.get_orchestration_configs(scenario='MyScenario', name='prompt_template_name', version='1.0.0')
print(response.resources)

Export Orchestration Config

Export a design orchestration config in a declarative compatible yaml file.

response = prompt_registry_client.export_orchestration_config(config_id=template_id)
print(response)