Caching Data in the Cloud¶

The service generator tool accepts an OData CSDL XML file (for OData version 2.0 or 4.0) as input and generates a Java web application project that can build a deployable WAR file for the implementation of an OData service. This document describes how to annotate the OData CSDL XML file to configure a Cache Database.

Note

Although the above section title refers to caching data in the cloud, generated services using cache databases also support on-premise deployment.

Cache Databases¶

The basic Conventions for OData Metadata allow you to create an OData service that is implemented atop of a SQL database. In that case, the SQL database can be considered to be a Back-end System.

It is sometimes helpful to synchronize data between an existing back-end system and a Cache Database. The generated OData service acts as an intermediary between client applications and the back-end system, and uses the cache database to add capabilities that might not be provided by the back-end system, such as OData change tracking.

A cache database is particularly useful for the support of occasionally-connected mobile client applications. Data from the back-end system is periodically synchronized (pulled or pushed) into the cache database, from where it can be downloaded by client applications. Subsequently, client applications can use OData delta links to efficiently download only changed data. Data changes can later be uploaded to the back-end system via the generated OData service, which treats the cache database as a write-through cache (but note that changes are only applied to the cache after being successfully applied to the back-end system).

Change Tracking¶

Cache databases rely upon the service generator's features for change tracking, which are separately documented.

Custom Annotations¶

The CSDL XML metadata for a cache database will include annotations using terms from the following annotation vocabularies:

The Cache vocabulary.
The SQL vocabulary.

To enable a cache database:

Annotate the EntityContainer with SQL.CacheDatabase to specify that the SQL database managed by the OData service will be a cache database.
Annotate the EntityContainer with SQL.TrackChanges to enable change tracking.
(Optionally) Annotate the EntityContainer with SQL.TrackDownloads to enable download tracking.
(Optionally) Enable client registrations with a ClientRegistration entity type and ClientRegistrationSet entity set.

Example:

<edmx:Edmx Version="4.0" xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://docs.oasis-open.org/odata/ns/edmx http://docs.oasis-open.org/odata/odata/v4.0/os/schemas/edmx.xsd http://docs.oasis-open.org/odata/ns/edm http://docs.oasis-open.org/odata/odata/v4.0/os/schemas/edm.xsd">
    <edmx:Reference Uri="vocabularies/com.sap.cloud.server.odata.sql.v1.xml">
        <edmx:Include Namespace="com.sap.cloud.server.odata.sql.v1" Alias="SQL"/>
    </edmx:Reference>
    <edmx:DataServices>
        <Schema Namespace="my.schema" Alias="Self" xmlns="http://docs.oasis-open.org/odata/ns/edm">
            <!-- ClientRegistration entity type omitted for brevity. -->
            ...
            <EntityContainer Name="MyService">
                <Annotation Term="SQL.CacheDatabase"/>
                <Annotation Term="SQL.TrackChanges"/>
                <Annotation Term="SQL.TrackDownloads"/>
                <!-- ClientRegistrationSet entity set omitted for brevity. -->
                ...
            </EntityContainer>
        </Schema>
    </edmx:DataServices>
</edmx:Edmx>

Client Applications¶

Because cache databases are usually used in support of offline-enabled clients, it is recommended that you build client applications using the SAP BTP SDK for Android or iOS together with Offline OData (the OfflineODataProvider API), which communicates with the generated OData service via the Offline OData server-side component in SAP Mobile Services. This combination enables optimized synchronization of changes between client applications and back-end systems.

Alternatively, use the CloudSyncProvider in the client SDK, which is more lightweight than OfflineODataProvider but is capable of directly accessing an mobile back-end tools-generated OData service and supporting change tracking.

When you use OfflineODataProvider or CloudSyncProvider together with an OData service for a cache database, please take particular note of the following topics which are important to understand with regard to establishing communication between the client application and the cache database.

Client Registrations (including the Client-Instance-ID header).
Client Credentials (including the X-Client-Credentials header).

Cached Entities¶

Cached entities are entities that will contain copies of data from a back-end system. The purpose of cached entities is to remove the burden of having to implement OData change tracking from the back-end system, or to remove the burden of having to implement change tracking in a way that supports complex selection criteria (see Download Queries).

`NoStore` Term¶

You can apply the Cache.NoStore term to an entity type to prevent entities of that type from being stored in the cache.

Example:

<EntityType Name="Stuff">
    <Annotation Term="Cache.NoStore"/>
    <Key>
        <PropertyRef Name="StuffID"/>
    </Key>
    <Property Name="StuffID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

All queries and create/update/delete operations for such an entity will be forwarded to the back-end system. The client will be responsible for dealing with the lack of change-tracking query responses.

`NoLoad` Term¶

You can apply the Cache.NoLoad term to an entity type or its entity set to prevent the entities from loading from the back-end system into the cache. Instead, the data is stored in the cache itself as the main source of information in response to client requests.

Example:

<EntityType Name="Stuff">
    <Annotation Term="Cache.NoLoad"/>
    <Key>
        <PropertyRef Name="StuffID"/>
    </Key>
    <Property Name="StuffID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

Data queries, along with create, update, and delete operations for these entities, aren't forwarded to the back-end system. The cache database serves as the persistent store for these entities.

`RefreshBy` Term¶

You can apply the Cache.RefreshBy term to an entity type to indicate how entities in the cache are to be updated. The value of this term is a string, as a comma-separated list, which may include dcn if the cache will be modified by Data Change Notification, and may also include either loadAll or loadPartition (see Cache.PartitionBy below).

Example:

<EntityType Name="Region">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Key>
        <PropertyRef Name="RegionID"/>
    </Key>
    <Property Name="RegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

Note

Any entity type without customized entity handlers and that does not use dcn will be stored directly in the cache database and will not be considered as a cached entity since it will be assumed that the entity type does not exist in any back-end system. Any such entity type can still be downloaded and uploaded by client applications.

`PartitionBy` Term¶

You can apply the Cache.PartitionBy term to an entity type to indicate that entities in the cache should be refreshed one partition at a time (using loadPartition). For example, perhaps the back-end system has no operation that can retrieve all Customer entities in one call, but instead can only retrieve the customer entities for a particular Region (entity type) by RegionID (foreign key property). Then you could partition the customer entity type by Region/RegionID.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="Region/RegionID"/>
    <Key>
        <PropertyRef Name="CustomerID"/>
    </Key>
    <Property Name="CustomerID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="RegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

The source partitioning item (e.g. Region/RegionID) must have a corresponding property (e.g. RegionID) in the target entity type (e.g. Customer). If the corresponding target property has a different name, e.g. MyRegionID, then the partitioning item syntax should be prefixed with an assignment to the target property.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="MyRegionID = Region/RegionID"/>
    <Key>
        <PropertyRef Name="CustomerID"/>
    </Key>
    <Property Name="CustomerID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="MyRegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

Immutable Partitioning Properties¶

The partitioning properties should be immutable, perhaps a subset (if composite) or a prefix of the key of the entity type that defines the partitions.

If partitioning properties are mutable, then cache refresh may not work correctly when an entity moves from one partition to another due to a change in the value of a partitioning property. This isn't a consideration when loadAll is used (non-partitioned caches), because the cache refresh for loadAll is able to process all of the back-end data for an entity set in a single step.

Partitioning for Memory Efficiency¶

When a loadAll or loadPartition call to the back-end system may return a very large set of data, that temporarily needs to be held in the OData service's JVM heap memory until cache merging is complete. Depending on the available JVM heap size, that might be too much data to hold in JVM heap memory at one time.

Consider whether smaller partitions may need to be carefully defined to avoid overloading the JVM memory. For example, if the back-end system has a small number of regions, a prefix of the region's key might be appropriate to increase the number of partitions, thereby making each partition smaller.

Another option to avoid JVM memory overload when using very large data sets is to populate the cache using dcn.

Partitioning by Multiple Properties¶

If you need to uniquely identify a partition using multiple properties of the partitioning entity, you should provide them as a comma-separated list.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="Region/City, Region/State"/>
    ...
</EntityType>

Partitioning by Client Registration¶

If you need to partition an entity type by client registration, use client as the value for the Cache.PartitionBy annotation.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="client"/>
    ...
</EntityType>

A shorthand Cache.ByClient term can be used in place of Cache.RefreshBy of loadPartition and Cache.PartitionBy of client.

<EntityType Name="Customer">
    <Annotation Term="Cache.ByClient"/>
    ...
</EntityType>

You can also partition by client in combination with regular partitioning properties.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="client, Region/RegionID"/>
    ...
</EntityType>

Note

Partitioning by client registration can result in multiple copies of back-end entities (i.e. one copy per registered client) being present in the cache database. This can be convenient if the back-end system has an existing operation that returns client-specific data. For data that is expected to be shared between many clients (such as reference/master data), partitioning by client registration is not recommended as it can result in excessive utilization of storage space within the cache database.

Partitioning by Client Language¶

If you need to partition an entity type by client language (see Accept-Language header), use locale as the value for the Cache.PartitionBy annotation.

Example:

<EntityType Name="Product">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="locale"/>
    ...
</EntityType>

A shorthand Cache.ByLocale term can be used in place of Cache.RefreshBy of loadPartition and Cache.PartitionBy of locale.

<EntityType Name="Product">
    <Annotation Term="Cache.ByLocale"/>
    ...
</EntityType>

You can also partition by locale in combination with regular partitioning properties.

Example:

<EntityType Name="Product">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="locale, CategoryID"/>
    ...
</EntityType>

When to Refresh¶

For an entity type that uses loadAll or loadPartition (in addition to or instead of dcn), one of the following annotations should be used to indicate how often (and when) the cache for that entity type will be refreshed.

The Cache.Schedule term is used to indicate that the cached entities for an entity type should be refreshed at a certain time of day (with UTC time, not local time), expressed as an Edm.TimeOfDay which uses the lexical representation of XML Schema time.

Example:
```
<EntityType Name="Region">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Annotation Term="Cache.Schedule" TimeOfDay="02:00:00"/>
    ...
</EntityType>
```
If a scheduled cache refresh fails to run successfully, it can be manually initiated by sending an HTTP GET request to the relevant entity set, appending "/$count?refresh-cache=true" to the usual URL. The user who invokes such a request must have the CacheAdministrator security role. Add a Cache-Controlheader if any intermediate proxy servers may have cached a previous response for this URL.

Example:
```
GET /Customers/$count?refresh-cache=true HTTP/1.1
Cache-Control: no-store, max-age=0
```
The Cache.Timeout term is used to indicate that the cached entities for an entity type should be refreshed at a specified interval, expressed as an Edm.Duration which uses the lexical representation of XML Schema dayTimeDuration. If not specified, the default timeout is one hour (except if Cache.Schedule was specified, in which case the refresh is performed once daily).

Example:
```
<EntityType Name="Region">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Annotation Term="Cache.Timeout" Duration="PT4H"/>
    ...
</EntityType>
```
The Cache.LoadAfter term is used to indicate that the cached entities for an entity type should be refreshed after the entities for some other entity type. This will often be the case for partitioned entity types, and also for child entity types in a parent/child relationship. If the server is deployed in a cluster (i.e. multiple server processes on one or more hosts), then Cache.LoadAfter ensures that the following cache refreshes (e.g. Customer in the below example) will be executed by the same process as the preceding cache refreshes (e.g. Region).

Example:
```
<EntityType Name="Customer">
    <Annotation Term="Cache.LoadAfter" String="Region"/>
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="Region/RegionID"/>
    ...
</EntityType>
```
The Cache.LoadBefore term is used to indicate that the cached entities for an entity type should be refreshed before the entities for one or more other entity types. This is effectively the inverse of Cache.LoadAfter. If the server is deployed in a cluster (i.e. multiple server processes on one or more hosts), then Cache.LoadBefore allows the possibility that the following cache refreshes (e.g. Customer and Supplier in the below example) may be executed concurrently by separate processes (running on separate hosts).

Example:
```
<EntityType Name="Region">
    <Annotation Term="Cache.LoadBefore" String="Customer, Supplier"/>
    ...
</EntityType>
```
The Cache.OnDemand term is used to indicate that the cached entities for an entity type should be refreshed right before a client's download query is executed, subject also to Cache.Timeout, which defaults to one hour if not specified. Short timeouts (such as PT0S to force cache refresh on each client download) should be used with care since, if clients frequently execute download queries, the resulting load on the back-end system and/or cache database may be excessive. However if the client always needs the most up-to-date data, PT0S may be appropriate.

Example:
```
<EntityType Name="CustomerVisit">
    <Annotation Term="Cache.OnDemand"/>
    ...
</EntityType>
```
The Cache.OnStartup term is used to indicate that the cached entities for an entity type should be refreshed at server startup time. This can be convenient in a test system (combined with either Cache.Schedule or Cache.Timeout) so that testers do not need to wait for a specified time of day or elapsed time interval before testing that data is being refreshed correctly in the cache database.

Example:
```
<EntityType Name="Region">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Annotation Term="Cache.OnStartup"/>
    <Annotation Term="Cache.Timeout" Duration="PT1H"/>
    ...
</EntityType>
```

When choosing between Cache.Schedule and Cache.Timeout, consider using Cache.Schedule (for daily refresh) to schedule cache refreshes for a time of day when not many clients are expected to be connected. This is particularly useful for entity sets that contain millions of entities. Consider using Cache.Timeout (e.g. for hourly refresh) when data must be more up-to-date for clients than daily scheduling would permit. However, note that timeout-based refresh may then occur during busy periods of client download activity. Because both cache refreshes and client downloads place a burden on the database server, configuring the timing of cache refreshes and client downloads to separate those activities may be helpful to overall system performance. If cache refreshes and client downloads are not separated, then the database server may need additional resources (such as CPU and RAM) to ensure that the impact on client download performance is acceptable to users.

An alternative to Cache.Schedule and Cache.Timeout (or even supplementing them) is to use DCN to refresh data in the cache database in response to change notifications from the back-end system. DCN usually reduces the workload of both the cache database and the back-end system, as compared with relying on schedules or timeouts. This is because schedules and timeouts are pull-based solutions that aren't aware of whether back-end system data has actually changed, whereas DCN is a push-based solution, which can be responsive to actual back-end changes.

Client User Propagation¶

If an entity's cache needs to be refreshed during a client's download request, but the entity type is not partitioned by client or locale, the load operation will be issued with technical user credentials unless the entity type is annotated with the Cache.PropagateUser term, in which case the user's credentials will be propagated (in accordance with the destination configuration).

Example:

<EntityType Name="Task">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Annotation Term="Cache.PropagateUser"/>
    ...
</EntityType>

The rationale for using technical user credentials by default (when the Cache.PropagateUser annotation is not specified), is that data that isn't specific to a particular client or locale is likely to be shared data. Populating shared data using a non-technical user's identity might be inappropriate.

Client-Requested Refresh¶

Apart from the refresh-cache=true URL query parameter (mentioned above) which might be used in a test system (or by a user with CacheAdministrator role), it may occasionally be useful to allow client applications to explicitly request the refreshing of the cache for one or more entity sets (or individual partitions if applicable).

Via customization of the ProviderSettings.init(MainServlet servlet) method, the DataServlet.setAllowXCacheRefresh method can be called to permit clients to send an X-Cache-Refresh header with their HTTP GET download requests, to specify the names of entity sets that should permit on-demand refresh, even if they are configured for scheduled/timeout-based refresh. By default, no entity sets will permit this kind of on-demand refresh.

The X-Cache-Refresh header value (if provided by a client) should be a comma separated list of items of the form EntitySetName;param1=value1;param2=value2;..., where the named parameters paramX=valueX;... are optional. This syntax for header values is similar to that used by the HTTP Accept header. Any commas within a list item must be percent-encoded. The list may contain a single item (if only one entity set's cache is to be on-demand refreshed), or multiple items (if multiple entity sets' caches are to be on-demand refreshed). A special parameter form partition=PartitionSet(PartitionKey) specifies a cache partition when the handler uses loadPartition, e.g. CustomerSet;partition=RegionSet(RegionID=123). The entity set to be refreshed, optionally with partition, and any other named parameters will be made available to a loadAll or loadPartition call via the DataQuery.customOptions (in the load function's query parameter). For example, the information from an X-Cache-Refresh header of CustomerSet;partition=RegionSet(RegionID=123);other=ABC will be passed to the handler's load function as customOptions map: {"refresh":"CustomerSet", "partition":"RegionSet(RegionID=123)", "other":"ABC"}.

Cache Partition Information¶

The cache database includes a table named xs_cache_partitions, which retains statistical and status/error information obtained during the last cache refresh for each cache or partition. You might find this information useful for capacity planning purposes, or for tuning/optimization.

See also: Monitoring Generated Services, for statistical aggregates.

Entity Handlers¶

Customizing Generated Services describes how you can use handler classes to customize the implementation of OData CRUD operations (create / read / update / delete) for OData entity types.

When you implement a cache database, you can implement the following handler methods for each entity type to interact with the back-end system.

loadAll, which should query the back-end system for all entities of a particular type and return those entities so that the caching system can merge any new or changed entities into the cache database (and delete any entities which are no longer present in the back-end system from the cache database).
loadPartition, which should query the back-end system for some entities of a particular type (as indicated by the Cache.PartitionBy term), and return those entities so that the caching system can merge any new or changed entities into the cache database (and delete any entities which are no longer present in the back-end system from the cache database).
createEntity, which should create an entity in the back-end system, obtain the (possibly generated) key properties and (possibly altered) non-key properties, and then create a corresponding entity in the cache database.
updateEntity, which should update an entity in the back-end system, obtain the (possibly altered) non-key properties, and then update the corresponding entity in the cache database.
deleteEntity, which should delete an entity from the back-end system, and then delete the corresponding entity from the cache database.

You can customize the generated handler classes to interact with arbitrary back-end systems, or in certain cases (for back-ends accessible via HTTP, JDBC or RFC) they can be generated automatically using the annotation terms Cache.LoadHandler, Cache.CreateHandler, Cache.UpdateHandler or Cache.DeleteHandler (see below for term descriptions).

Merge-Based Refresh¶

The default mechanism used by the server runtime when refreshing a cache (or a partition) is:

Call the handler's loadAll (or loadPartition) method to get obtain the relevant list of entities in the back-end system.
Call the handler's executeQuery method to obtain the relevant list of entities in the cache database.
Perform an in-memory sort of both lists.
Perform an in-memory merge of the sorted lists to determine which inserts, updates and deletes need to be applied to the cache database.

This process can be memory-insensitive. Size the JVM heap accordingly. Consider whether partitioning, DCN, or one of the following two optimizations may be appropriate alternatives.

Delta-Link Optimization¶

The merge-based refresh mechanism can be streamlined if the handler's loadAll (or loadPartition) method can (when the cache is initially populated) include a deltaLink in the EntityValueList it returns. Subsequent refreshes will pass that delta link in the query parameter (see the deltaLink and deltaToken properties in the DataQuery class), and the load method is then expected to return only the back-end changes since the point-in-time indicated by that delta link.

Set the EntityValue.isDeleted to true for any entity that has been deleted from the back-end system and therefore needs to be deleted from the cache, otherwise any entity returned in the load method's result will be treated as an upsert to the cache.

Pull-Push Optimization¶

When the handler's loadAll (or loadPartition) method is called to pull data from the back-end system, it may elect to explicitly push relevant changes into the cache, bypassing the usual merge or delta handling described above.

The load method can create a list of entities (each of which may be marked isDeleted or not as appropriate), then pass that list of entities in a call to servlet.dcnTransaction(entities). This may be repeated any number of times, each dcnTransaction call providing an additional batch of changed entities. Using multiple dcnTransaction calls can be an effective memory optimization as it will reduce the in-memory size of each entity batch.

The returned EntityValueList has a property loadedByDCN which the load method should set to EntityValueList.DCN_FULL_REFRESH or EntityValueList.DCN_PARTIAL_REFRESH depending on whether the entire cache (or partition) was updated by the dcnTransaction call(s). If the loadedByDCN property is not set (or is set to zero), the server runtime will assume it needs to use the default merge or delta handling, as appropriate.

Example pseudo-code for pull-push optimization:

public void loadAll(DataQuery query) {
    EntityValueList changes = new EntityValueList();
    ... // start getting changes from back-end
    for (;;) {
        EntityValue entity = nextBackendChange();
        if (entity == null) break;
        // entity.setDeleted(true); // if appropriate
        changes.add(entity);
        if (changes.length() == 1000) {
            servlet.dcnTransaction(changes);
            changes.clear();
        }
    }
    servlet.dcnTransaction(changes); // last batch might be less than 1000
    EntityValueList result = new EntityValueList();
    result.setLoadedByDCN(EntityValueList.DCN_FULL_REFRESH);
    return result;
}

OData Back-End Systems¶

If the back-end system being cached is itself an OData service, then entity handlers can be automatically generated by using a single Cache.ODataBackend annotation within the EntityContainer, together with the appropriate annotations for cached entities such as Cache.RefreshBy, Cache.PartitionBy etc.

Example:

<edmx:Edmx Version="4.0"
    xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://docs.oasis-open.org/odata/ns/edmx http://docs.oasis-open.org/odata/odata/v4.0/os/schemas/edmx.xsd http://docs.oasis-open.org/odata/ns/edm http://docs.oasis-open.org/odata/odata/v4.0/os/schemas/edm.xsd">
    <edmx:Reference Uri="vocabularies/com.sap.cloud.server.odata.sql.v1.xml">
        <edmx:Include Namespace="com.sap.cloud.server.odata.sql.v1" Alias="SQL"/>
    </edmx:Reference>
    <edmx:DataServices>
        <Schema Namespace="my.schema" Alias="Self" xmlns="http://docs.oasis-open.org/odata/ns/edm">
            ...
            <EntityContainer Name="MyService">
                <Annotation Term="Cache.ODataBackend" String="destination-name"/>
                <Annotation Term="SQL.CacheDatabase"/>
                <Annotation Term="SQL.TrackChanges"/>
                <Annotation Term="SQL.TrackDownloads"/>
                ...
            </EntityContainer>
        </Schema>
    </edmx:DataServices>
</edmx:Edmx>

The Cache.ODataBackend term specifies an HTTP destination name for the OData back-end system.

It is not mandatory to use this technique when caching an OData back-end system. Manually coded entity handlers or Cache.LoadHandler annotations can be used instead. However using the Cache.ODataBackend term is likely to require considerably less developer effort.

Tip

When caching an OData back-end system, the use of Externalized Annotations and Externalized Definitions (where appropriate) is recommended, as in that case the main metadata file can be a verbatim copy of the OData back-end's metadata. This means that if the back-end's metadata changes (e.g. with the addition of entites or properties), it won't be necessary to manually merge the annotations needed for caching into the copied main metadata file.

Entity Handlers for HTTP Destinations¶

The Cache.HttpDestination term is used to specify the destination name of a back-end system that is accessed using HTTP or HTTPS. Such a system would normally be a RESTful Web Service using JSON or XML for request and response payloads.

The Cache.HttpDestination term is placed within the EntityContainer element (if all HTTP operations use the same destination), or within each relevant EntityType element (if there are multiple HTTP destinations).

Example:

<EntityContainer Name="MyService">
    <Annotation Term="Cache.HttpDestination" String="destination-name"/>
    ...
</EntityContainer>

The connectivity information for HTTP destinations is defined externally to the metadata. See Configuring Destinations for details.

The cache handler annotation terms (Cache.LoadHandler, Cache.CreateHandler, Cache.UpdateHandler, Cache.DeleteHandler) are used together with the HttpRequest property providing the method and URL for an HTTP request (see entity bindings for URL).

The other HTTP handler properties (RequestHeaders, RequestBody, ResponseHeaders, ResponseBody) are used as required with entity bindings using JSON or XML that match headers and content from requests and responses with entity properties.

The HTTP Content-Type and Accept headers can be determined automatically from the entity bindings, if the entity bindings use JSON or XML and the corresponding expected content type is application/json or application/xml. So using the RequestHeaders property is often not required.

If the backend system is an OData service, and instead of using the Cache.ODataBackend you are using Cache.*Handler annotations, the DataServiceVersion or OData-Version header should be specified (as appropriate to the OData version of the back-end system).

Note

Important: a POST (create) operation for an OData entity is required to produce a Location header in the response (from the generated OData service), so if the primary key of the created entity is not provided by the client application but rather is generated by the back-end system, a response binding is required for Cache.CreateHandler to bind the back-end-generated key to the entity's key property.

Example:

<EntityType Name="Region">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Key>
        <PropertyRef Name="RegionID"/>
    </Key>
    <Property Name="RegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    <Property Name="Info" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
    <Annotation Term="Cache.LoadHandler">
        <Record>
            <PropertyValue Property="HttpRequest" String="GET /region"/>
            <PropertyValue Property="ResponseBody">
                <String>
                    [
                        {
                            "id": "${entity.RegionID}",
                            "name": "${entity.Name}"
                        }
                    ]
                </String>
            </PropertyValue>
        </Record>
    </Annotation>
    <Annotation Term="Cache.CreateHandler">
        <Record>
            <PropertyValue Property="HttpRequest" String="POST /region"/>
            <PropertyValue Property="RequestHeaders">
                <String>
                    {
                        "Accept": "application/json"
                    }
                </String>
            </PropertyValue>
            <PropertyValue Property="RequestBody">
                <String>
                    {
                        "name": "${entity.Name}"
                    }
                </String>
            </PropertyValue>
            <PropertyValue Property="ResponseHeaders">
                <!-- Note: it is unusual to bind response headers to non-key entity properties -->
                <String>
                    {
                        "X-Info": "${entity.Info}"
                    }
                </String>
            </PropertyValue>
            <PropertyValue Property="ResponseBody">
                <!-- Must bind the back-end-generated key to the entity's key property -->
                <String>
                    {
                        "id": "${entity.RegionID}"
                    }
                </String>
            </PropertyValue>
        </Record>
    </Annotation>
    <Annotation Term="Cache.UpdateHandler">
        <Record>
            <PropertyValue Property="HttpRequest" String="PUT /region/${entity.RegionID}"/>
            <PropertyValue Property="RequestBody">
                <String>
                    {
                        "name": "${entity.Name}"
                    }
                </String>
            </PropertyValue>
        </Record>
    </Annotation>
    <Annotation Term="Cache.DeleteHandler">
        <Record>
            <PropertyValue Property="HttpRequest" String="DELETE /region/${entity.RegionID}"/>
        </Record>
    </Annotation>
</EntityType>

Entity Handlers for RFC Destinations¶

The Cache.RfcDestination term is used to specify the destination name of a back-end system that is accessed using SAP Java Connector.

The Cache.RfcDestination term is placed within the EntityContainer element (if all RFC operations use the same destination), or within each relevant EntityType element (if there are multiple RFC destinations).

Example:

<EntityContainer Name="MyService">
    <Annotation Term="Cache.RfcDestination" String="destination-name"/>
    ...
</EntityContainer>

The connectivity information for RFC destinations is defined externally to the metadata. See Configuring Destinations for details.

For best performance, especially for Cache.LoadHandler calls that may produce large results, consult the JCo documentation regarding Fast RFC Serialization.

The cache handler annotation terms (Cache.LoadHandler, Cache.CreateHandler, Cache.UpdateHandler, Cache.DeleteHandler) are used together with the RfcFunction property providing the name of an RFC function.

The other RFC handler properties (InputRecord, OutputRecord) are used as required with entity bindings using JSON that match RFC input and output parameters from requests and responses with entity properties.

Note

Important: a POST (create) operation for an OData entity is required to produce a Location header in the response (from the generated OData service), so if the primary key of the created entity is not provided by the client application but rather is generated by the back-end system, a response binding is required for Cache.CreateHandler to bind the back-end-generated key to the entity's key property.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Key>
        <PropertyRef Name="CustomerID"/>
    </Key>
    <Property Name="City" Type="Edm.String" Nullable="true" MaxLength="100"/>
    <Property Name="Country" Type="Edm.String" Nullable="true" MaxLength="10"/>
    <Property Name="CountryISO" Type="Edm.String" Nullable="true" MaxLength="10"/>
    <Property Name="CustomerID" Type="Edm.String" Nullable="false" MaxLength="8"/>
    <Property Name="Email" Type="Edm.String" Nullable="true" MaxLength="100"/>
    <Property Name="Honorific" Type="Edm.String" Nullable="true" MaxLength="20"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="100"/>
    <Property Name="Phone" Type="Edm.String" Nullable="true" MaxLength="50"/>
    <Property Name="POBox" Type="Edm.String" Nullable="true" MaxLength="20"/>
    <Property Name="PostCode" Type="Edm.String" Nullable="true" MaxLength="20"/>
    <Property Name="Region" Type="Edm.String" Nullable="true" MaxLength="100"/>
    <Property Name="Street" Type="Edm.String" Nullable="true" MaxLength="100"/>
    ...
    <Annotation Term="Cache.LoadHandler">
        <Record>
            <PropertyValue Property="RfcFunction" String="BAPI_FLBOOKING_GETLIST"/>
            <PropertyValue Property="OutputRecord">
                <!-- Note: a JSON array binding is used for a JCo table parameter -->
                <String>
                    {
                        "CUSTOMER_LIST":
                        [
                            {
                                "CITY": "${entity.City}",
                                "COUNTR": "${entity.Country}",
                                "COUNTR_ISO": "${entity.CountryISO}",
                                "CUSTNAME": "${entity.Name}",
                                "CUSTOMERID": "${entity.CustomerID}",
                                "EMAIL": "${entity.Email}",
                                "FORM": "${entity.Honorific}",
                                "PHONE": "${entity.Phone}",
                                "POBOX": "${entity.POBox}",
                                "POSTCODE": "${entity.PostCode}",
                                "REGION": "${entity.Region}",
                                "STREET": "${entity.Street}"
                            }
                        ]
                    }
                </String>
            </PropertyValue>
        </Record>
    </Annotation>
    <Annotation Term="Cache.CreateHandler">
        <Record>
            <PropertyValue Property="RfcFunction" String="BAPI_FLCUST_CREATEFROMDATA"/>
            <PropertyValue Property="InputRecord">
                <!-- Note: a JSON object binding is used for an RFC structure parameter -->
                <String>
                    {
                        "CUSTOMER_DATA":
                        {
                            "CITY": "${entity.City}",
                            "COUNTR": "${entity.Country}",
                            "COUNTR_ISO": "${entity.CountryISO}",
                            "CUSTNAME": "${entity.Name}",
                            "CUSTTYPE": "B",
                            "EMAIL": "${entity.Email}",
                            "FORM": "${entity.Honorific}",
                            "PHONE": "${entity.Phone}",
                            "POBOX": "${entity.POBox}",
                            "POSTCODE": "${entity.PostCode}",
                            "REGION": "${entity.Region}",
                            "STREET": "${entity.Street}"
                        }
                    }
                </String>
            </PropertyValue>
            <PropertyValue Property="OutputRecord">
                <!-- Note: a JSON field binding is used for an RFC primitive parameter -->
                <String>
                    {
                        "CUSTOMERNUMBER": "${entity.CustomerID}"
                    }
                </String>
            </PropertyValue>
        </Record>
    </Annotation>
</EntityType>

Client Credentials for RFC Destinations¶

When working with a cache database, there are two kinds of thread that may make requests to the back-end system:

A foreground thread which is servicing a request from a client application, for example propagating a createEntity request to the back-end system, or performing on-demand refresh of a cache due to use of the Cache.OnDemand annotation.
A background thread which is executing a request to the back-end system on behalf of multiple users, such as when refreshing shared entities in cache for an entity type using a Cache.Schedule annotation.

It may be desirable to have foreground threads propagate some credentials from the client application through to the back-end system. This is achieved by specifying a Cache.JCoCredentials annotation within the EntityContainer element, using client credentials bindings to provide values for JCo user logon properties.

Note

This feature is only currently available for on-premise OData services.

Example:

<EntityContainer Name="MyService">
    <Annotation Term="Cache.JCoCredentials">
        <String>
            {
                "jco.client.user": "${client.backendUsername}",
                "jco.client.passwd": "${client.backendPassword}"
            }
        </String>
    </Annotation>
    ...
</EntityContainer>

The corresponding RFC destination properties need to include the value CURRENT_USER for the jco.destination.auth_type property.

Example:

jco.destination.auth_type=CURRENT_USER
jco.client.serialization_format=columnBased
jco.client.type=3
jco.client.ashost=<the-ashost>
jco.client.r3name=<the-r3name>
jco.client.sysnr=00
jco.client.client=900
jco.client.user=<some-technical-username>
jco.client.passwd=<some-technical-password>

Note

In the above example, values are provided for jco.client.user and jco.client.passwd even though the Cache.JCoCredentials annotation has been used to provide client credentials bindings. Values from the corresponding RFC destination properties will be used by background threads, and values from the client credentials bindings will be used by foreground threads.

Entity Handlers for SQL Destinations¶

The Cache.SqlDestination term is used to specify the destination name of a back-end system that is accessed using JDBC.

The Cache.SqlDestination term is placed within the EntityContainer element (if all RFC operations use the same destination), or within each relevant EntityType element (if there are multiple RFC destinations).

Example:

<EntityContainer Name="MyService">
    <Annotation Term="Cache.SqlDestination" String="destination-name"/>
    ...
</EntityContainer>

The connectivity information for SQL destinations is defined externally to the metadata. See Configuring Destinations for details.

The cache handler annotation terms (Cache.LoadHandler, Cache.CreateHandler, Cache.UpdateHandler, Cache.DeleteHandler) are used together with the SqlStatement property providing the text of an SQL statement with entity bindings for SQL.

Note

Important: a POST (create) operation for an OData entity is required to produce a Location header in the response (from the generated OData service), so if the primary key of the created entity is not provided by the client application but rather is generated by the back-end system, a response binding is required for Cache.CreateHandler to bind the back-end-generated key to the entity's key property.

Entity Bindings¶

A back-end operation may expect input parameters or return output parameters that need to be bound to entity properties when an entity handler method (loadAll, loadPartition, createEntity, updateEntity, or deleteEntity) is called. Operation parameters are matched with entity properties using entity bindings.

A regular entity binding is a string of the form ${entity.PropertyName}. It can be used for binding input or output parameters to the properties of the entity type within which the binding occurs.

An old entity binding is a string of the form ${old.PropertyName}. It can be used for binding input parameters to the old properties of the entity type within which the binding occurs. This is applicable to updateEntity and deleteEntity, and may be helpful for the avoidance of lost updates using optimistic concurrency control techniques.

A partition entity binding is a string of the form ${partition.PropertyName}. It can be used for binding input parameters to the properties of the partition type for an entity type which defines a loadPartition method, which will associate the input parameters with the property values corresponding to the partition which is currently being loaded into the cache database.

A client credentials binding is a string of the form ${client.PropertyName}. It can be used for binding input parameters to the properties of the ClientCredentials entity type, which will associate the input parameters with the property values corresponding to the client which is currently executing the request that is being delegated to the entity handler.

A client registration binding is a string of the form ${client.PropertyName}. It can be used for binding input parameters to the properties of the ClientRegistration entity type, which will associate the input parameters with the property values corresponding to the client which is currently executing the request that is being delegated to the entity handler.

A header structure binding is a string of the form ${header.StructureName.PropertyName}. It can be used to obtain parameter values from an HTTP header. Custom HTTP headers may be used for passing parameters other than entity properties. For example, within an entity type Customer, the binding ${header.CreateParameters.Reason} would expect an HTTP header X-Create-Parameters whose value is a data URI containing the Base64-encoded JSON representation of a CustomerCreateParameters complex value containing a Reason property. To facilitate this encoding of credentials, see ToJSON.dataURI and HttpHeaders.withData in the client SDK documentation, or refer to the following example showing the encoding steps.

Example encoding of parameters object with Reason of "special":

JSON-encoded object: {"Reason":"special"}
Base64-encoded JSON: eyJSZWFzb24iOiJzcGVjaWFsIn0=
Encoded as data URI: data:application/json;base64,eyJSZWFzb24iOiJzcGVjaWFsIn0=
HTTP header: X-Create-Parameters: data:application/json;base64,eyJSZWFzb24iOiJzcGVjaWFsIn0=

Note

Header structure bindings are provided to support interoperation with back-end systems whose operations may expect some parameters that do not exist as properties in the back-end entity model. It is preferable to use regular entity bindings wherever possible.

Entity Bindings for JSON¶

JSON entity bindings use a String containing the text for a JSON array or JSON object, possibly containing nested JSON arrays or objects, to arbitrary levels of nesting as required by the back-end operation. The ${entity.PropertyName} syntax is used within field values inside JSON objects.

Example:

<Annotation Term="Cache.UpdateHandler">
    <Record>
        <PropertyValue Property="HttpRequest" String="PUT /patient/${entity.PatientID}"/>
        <PropertyValue Property="RequestBody">
            <String>
                {
                    "name": "${entity.Name}",
                    "address": "${entity.Address}",
                    "dateOfBirth": "${entity.DOB}"
                }
            </String>
        </PropertyValue>
    </Record>
</Annotation>

Since JSON bindings are contained inside a CSDL XML document, it may sometimes be convenient to use CDATA sections so that XML special characters (less-than "<", greater-than ">", ampersand "&"), which may occur within the JSON bindings, do not need to be escaped using XML entity references.

Example:

<Annotation Term="Cache.UpdateHandler">
    <Record>
        <PropertyValue Property="HttpRequest" String="PUT /patient/${entity.PatientID}"/>
        <PropertyValue Property="RequestBody">
            <String><![CDATA[
                {
                    "name": "${entity.Name}",
                    "address": "${entity.Address}",
                    "dateOfBirth": "${entity.DOB}",
                    "special_<&>": "${entity.SpecialInfo}"
                }
            ]]></String>
        </PropertyValue>
    </Record>
</Annotation>

For binding results of a loadAll or loadPartition call, if the JSON bindings include multiple arrays, then one of those arrays needs to be designated as the one whose occurrences determine the number of result entities. By default the innermost nested array will implicitly designate the number of result entities, but it can be explicitly designated using a special first item within the array containing the string "for (entity of result)". This syntax is modeled after JavaScript Array comprehensions but is quoted within a string to be JSON-compatible.

Example:

<Annotation Term="Cache.LoadHandler">
    <Record>
        <PropertyValue Property="HttpRequest" String="GET /order"/>
        <PropertyValue Property="ResponseBody">
            <String>
                {
                    "results":
                    [
                        {
                            "orderId": "${entity.OrderID}"
                            "items":
                            [
                                "for (entity of result)",
                                {
                                    "itemId": "${entity.ItemID}",
                                    "quantity": ${entity.Quantity}"
                                }
                            ]
                        }
                    ]
                }
            </String>
        </PropertyValue>
    </Record>
</Annotation>

Two additional special forms of JSON entity binding may be useful for processing the responses from some RESTful APIs.

An object field can be bound to @nextLink if the back-end system breaks query responses into separate linked documents.
An entity binding can be placed after a parenthesized group within a regular expression to extract only a portion of a JSON field value into an entity property.

Example:

<Annotation Term="Cache.LoadHandler">
    <Record>
        <PropertyValue Property="HttpRequest" String="GET /region"/>
        <PropertyValue Property="ResponseBody">
            <!-- Back-end system returns a series of documents with next links -->
            <!-- Back-end system returns URLs (e.g. "http://example.com/region/123/") rather than simple IDs -->
            <String>
                {
                    "next": "@nextLink",
                    "results":
                    [
                        {
                            "url": ".*/(\\d+)${entity.RegionID}/",
                            "name": "${entity.Name}"
                        }
                    ]
                }
            </String>
        </PropertyValue>
    </Record>
</Annotation>

Entity Bindings for XML¶

XML entity bindings use a String containing the text for an XML document, possibly containing nested XML elements or attributes, to arbitrary levels of nesting, as required by the back-end operation. The ${entity.PropertyName} syntax is used within the text of XML elements and attributes.

Since XML bindings are contained inside a CSDL XML document, it is recommended to use CDATA sections so that XML special characters (less-than "<", greater-than ">", ampersand "&"), which often occur within the XML bindings, do not need to be escaped using XML entity references.

Example:

<Annotation Term="Cache.UpdateHandler">
    <Record>
        <PropertyValue Property="HttpRequest" String="PUT /patients"/>
        <PropertyValue Property="RequestBody">
            <String><![CDATA[
                <patient id="${entity.PatientID}">
                    <name>${entity.Name}</name>
                    <address>${entity.Address}</address>
                    <dateOfBirth>${entity.DOB}</dateOfBirth>
                <patient>
            ]]></String>
        </PropertyValue>
    </Record>
</Annotation>

For binding results of a loadAll or loadPartition call, one of the non-root elements should be designated as the one whose occurrences determine the number of result entities, using a special XML attribute: for-each="entity of result".

Example:

<Annotation Term="Cache.LoadHandler">
    <Record>
        <PropertyValue Property="HttpRequest" String="GET /patients"/>
        <PropertyValue Property="ResponseBody">
            <String><![CDATA[
                <patients>
                    <patient id="${entity.PatientID}" for-each="entity of result">
                        <name>${entity.Name}</name>
                        <address>${entity.Address}</address>
                        <dateOfBirth>${entity.DOB}</dateOfBirth>
                    <patient>
                </patients>
            ]]></String>
        </ProperyValue>
    </Record>
</Annotation>

Two additional special forms of XML entity binding may be useful for processing the responses from some RESTful APIs.

An XML element or attribute can be bound to @nextLink if the back-end system breaks query responses into separate linked documents.
An entity binding can be placed after a parenthesized group within a regular expression to extract only a portion of an XML element or attribute into an entity property.

Example:

<Annotation Term="Cache.LoadHandler">
    <Record>
        <PropertyValue Property="HttpRequest" String="GET /region"/>
        <PropertyValue Property="ResponseBody">
            <!-- Back-end system returns a series of documents with next links -->
            <!-- Back-end system returns URLs (e.g. "http://example.com/region/123/") rather than simple IDs -->
            <String><![CDATA[
                <mydoc next="@nextLink">
                    <region for-each="entity of result">
                        <url>.*/(\\d+)${entity.RegionID}/</url>
                        <name>${entity.Name}</name>
                    </region>
                </mydoc>
            ]]></String>
        </PropertyValue>
    </Record>
</Annotation>

Entity Bindings for SQL¶

JDBC entity bindings use a String containing the text for a SQL statement, as expected by the target database, expressed as Embedded SQL using :PropertyName host variable syntax instead of ${entity.PropertyName} for regular entity bindings, using :old.PropertyName for old entity bindings, using :partition.PropertyName for partition entity bindings, and using :client.PropertyName for client property bindings.

Since SQL bindings are contained inside a CSDL XML document, it may sometimes be convenient to use CDATA sections so that XML special characters (less-than "<", greater-than ">", ampersand "&"), which may occur within the SQL bindings, do not need to be escaped using XML entity references.

Example:

<EntityType Name="Patient">
    <Key>
        <PropertyRef Name="PatientID"/>
    </Key>
    <Property Name="PatientID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" MaxLength="100" Nullable="false"/>
    <Property Name="Address" Type="Edm.String" MaxLength="500" Nullable="false"/>
    <Property Name="DOB" Type="Edm.Date" Nullable="true"/>
    ...
    <Annotation Term="Cache.LoadHandler">
        <Record>
            <PropertyValue Property="SqlStatement">
                <String>
                    select id, name, address, date_of_birth
                    into :PatientID, :Name, :Address, :DOB
                    from patient
                </String>
            </PropertyValue>
        </Record>
    </Annotation>
    <Annotation Term="Cache.CreateHandler">
        <Record>
            <PropertyValue Property="SqlStatement">
                <String>
                    insert into patient (name, address, date_of_birth)
                    values (:Name, :Address, :DOB)
                    returning id
                </String>
            </PropertyValue>
        </Record>
    </Annotation>
    <Annotation Term="Cache.UpdateHandler">
        <Record>
            <PropertyValue Property="SqlStatement">
                <String>
                    update patient
                    set name = :Name, address = :Address, date_of_birth = :DOB
                    where id = :PatientID
                </String>
            </PropertyValue>
        </Record>
    </Annotation>
    <Annotation Term="Cache.DeleteHandler">
        <Record>
            <PropertyValue Property="SqlStatement">
                <String>
                    delete from patient where id = :PatientID
                </String>
            </PropertyValue>
        </Record>
    </Annotation>
</EntityType>

The provided SQL statement will not be passed directly to the back-end database. It will be preprocessed for use with a JDBC PreparedStatement:

All host variable references (e.g. :Name) will be replaced with a JDBC PreparedStatement parameter placeholder (?).
The into clause for select statements is provided to match result columns with entity properties. It will not be passed to the database.
Unless the client application is expected to provide the primary key, the returning clause for insert statements is required to indicate that a database-generated key will be returned. It will be transformed into the use of a database-specific mechanism (usually PreparedStatement.getGeneratedKeys()) for retrieving the generated key. For databases supporting key generation with identity columns, the key column does not need to be included in the values clause. For databases supporting key generation with sequence types, the values clause may need to include the database-specific syntax for obtaining the next value from the sequence.

Example:

<Annotation Term="Cache.CreateHandler">
    <Record>
        <PropertyValue Property="SqlStatement">
            <String>
                insert into patient (id, name, address, date_of_birth)
                values (patient_id_seq.nextval, :Name, :Address, :DOB)
                returning id
            </String>
        </PropertyValue>
    </Record>
</Annotation>

Entity Bindings for URL¶

URL entity bindings may appear within the String for an HttpRequest property within a cache handler annotation. The ${entity.PropertyName}, ${partition.PropertyName} and ${client.PropertyName} property binding forms can all be used.

Example:

<Annotation Term="Cache.UpdateHandler">
    <Record>
        <PropertyValue Property="HttpRequest" String="PUT /customer/${entity.CustomerID}"/>
        ...
    </Record>
</Annotation>

Additionally, an entity binding can be wrapped with encode(...) to percent-encode strings which might contain reserved URL characters.

Example:

<Annotation Term="Cache.UpdateHandler">
    <Record>
        <PropertyValue Property="HttpRequest" String="PUT /region/${encode(entity.Name)}"/>
        ...
    </Record>
</Annotation>

Data Change Notification¶

Previously it was described how to use a SQL.RefreshBy annotation with loadAll or loadPartition so that data will be pulled periodically from the back-end system into the cache database.

Another option, which can be more efficient if the back-end system has some form of built-in change logging, is to enhance the back-end system so that data changes will be pushed periodically into the cache database via the OData service. This is known as Data Change Notification (DCN).

To minimize unnecessary processing in the OData service, the back-end system should make some attempt to send only changed data in DCN messages, i.e. data that was changed in the back-end system after a previously successful DCN request.

DCN Batches¶

In order to reduce network latency, and considering that it is typical for multiple data changes from the back-end system to be pushed into the cache database around the same time, DCN requests must be sent in batches. A batch with only one change is acceptable, but batch sizes of 100 or more are recommended (unless the batches contain large data, such as images, in which case a smaller batch size, such as 10, is recommended).

A DCN batch must be sent to the URL dcn/$batch relative to the service root (a regular OData batch request would be sent to the URL $batch relative to the service root). Two DCN batch formats are supported:

OData JSON batch format using a method of put, patch or delete. The DCN sender should format the entities within the payload using OData 4.0 JSON representation, even if the service metadata uses a prior version.
SAP Mobile Platform MBO DCN format with payload, and an operation (op) of :upsert or :delete. This format is intended only for backwards compatibility with the SAP Mobile Platform MBO Runtime. New development should use the OData JSON batch format.

Note the following:

For efficiency, each DCN batch will be executed within a single database transaction (even for OData JSON batch format without atomicityGroup being specified).
Back-end systems initiating DCN requests may wish to have multiple processes or threads simultaneously processing non-overlapping subsets of the changed data to take advantage of the available capacity of multiprocessor systems. However, if DCN requests are likely to overlap in time with periods where clients are downloading changes, excessive parallelism of DCN requests may degrade client download performance. In all cases when using DCN, some benchmarking should be done to determine appropriate batch sizes and levels of parallelism.
OData JSON batch format expects the url field to include an entity set name (e.g. CustomerSet), whereas SAP Mobile Platform MBO DCN format expects the mbo field to include an entity type name (e.g. Customer).
DCN batches with OData JSON batch format cannot use the post method, because the back-end system will not usually be certain whether or not the cache database already contains a particular entity that has been changed. Allowing post could potentially result in many create conflicts (due to duplicate keys), likely causing poor system throughput. Thus the put method is used to implement upsert behavior.
DCN batches with OData JSON batch format can use the patch method if the back-end system knows it has previously put an entity and now wants to change a subset of its properties. However the implementation of patch will in most cases be slower than put, even though put must provide all properties, because put can make use of database-level batched updates, whereas patch uses a non-batched select-then-update strategy. Patching an entity which does not exist in the cache will result in status 404 (Not Found) for the corresponding item in the response payload.
DCN requests support the X-Versioning-Mode header. When you set it to skip-if-unchanged, the row_version doesn't increment if the incoming payload matches the record in the cache database. By default, the row_version increments with every update without checking the prior state. Avoid using skip-if-unchanged during initial cache loading, such as bulk DCN requests. The additional comparisons can significantly slow down the process.
Other than status of 404 (Not Found) for patching non-existing entities, each item in the response payload will indicate status of 204 (No Content) when using OData batch format, and success of true when using MBO DCN format.
If the enclosing HTTP POST response has a status other than 200 (OK), then the back-end system should assume that the DCN processing might not have completed successfully (due to database or network failure, for instance), in which case it should be repeated (POSTed again) after a suitable delay (see exponential backoff). If such failures are consistent rather than intermittent, it might indicate that the request payload is invalid (possibly containing invalid entity type, set, or property names), in which case the back-end system code issuing the DCN requests will need to be corrected. The embedded metrics service provides a dcnBatchFailures metric to facilitate checking for DCN failures.

DCN Examples¶

HTTP request message using OData JSON batch format:

POST /dcn/$batch HTTP/1.1
Accept: application/json
OData-Version: 4.0
Content-Type: application/json
Content-Length: ...

{
    "requests":
    [
        {"id": "1", "method": "put", "url": "Customers(123)",
            "body": {"Name": "Jean Williams", "Address": "33 Main St."}},
        {"id": "2", "method": "patch", "url": "Customers(456)",
            "body": {"Address": "25 Oak St."}},
        {"id": "3", "method": "delete", "url": "Customers(789)"}
    ]
}

HTTP response message using OData JSON batch format:

HTTP/1.1 200 OK
OData-Version: 4.0
Content-Type: application/json
Content-Length: ...

{
    "responses":
    [
        {"id": "1", "status": 204},
        {"id": "2", "status": 204},
        {"id": "3", "status": 204}
    ]
}

HTTP request message using SAP Mobile Platform MBO DCN format:

POST /dcn/$batch HTTP/1.1
Content-Type: application/json
Accept: application/json
Content-Length: ...

{
    "messages":
    [
        {"id": "1", "op": ":upsert", "mbo": "Customer",
            "cols": {"CustomerID": 123, "Name": "Jean Williams", "Address": "33 Main St."}},
        {"id": "2", "op": ":delete", "mbo": "Customer",
            "cols": {"CustomerID": 789}}
    ]
}

HTTP response message using SAP Mobile Platform MBO DCN format:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: ...

[
    {"recordID": "1", "success": true, "statusMessage": ""},
    {"recordID": "2", "success": true, "statusMessage": ""}
]

DCN Security¶

If the generated OData service is deployed with security enabled, then all DCN requests must be sent by an authenticated service user possessing the dcn security role.

Using the instructions in Configuring Destinations, configure an <appname>_dcn destination, including username/password credentials that will be required for any incoming DCN requests. By default, each incoming DCN batch request should have an Authorization header with Basic authentication.

You may also choose to register one of the following DCN authentication callbacks using the DataServlet API:

registerDcnPasswordPolicy - provide a callback that will check if the provided password matches some custom password policy. This policy check is in addition to checking that the incoming DCN request's password matches the password defined for the destination. The required password itself should not be included in custom code.
registerDcnCertificateChecker - provide a callback that will check the incoming DCN request headers for appropriate values in the X-Forwarded-Client-Cert header or other appropriate headers (instead of expecting Basic authentication). See Mutual TLS Authentication (mTLS) and Certificates Handling.

Last update: November 20, 2025

Caching Data in the Cloud¶

Cache Databases¶

Change Tracking¶

Custom Annotations¶

Client Applications¶

Cached Entities¶

NoStore Term¶

NoLoad Term¶

RefreshBy Term¶

PartitionBy Term¶

Immutable Partitioning Properties¶

Partitioning for Memory Efficiency¶

Partitioning by Multiple Properties¶

Partitioning by Client Registration¶

Partitioning by Client Language¶

When to Refresh¶

Client User Propagation¶

Client-Requested Refresh¶

Cache Partition Information¶

Entity Handlers¶

Merge-Based Refresh¶

Delta-Link Optimization¶

Pull-Push Optimization¶

OData Back-End Systems¶

Entity Handlers for HTTP Destinations¶

Entity Handlers for RFC Destinations¶

Client Credentials for RFC Destinations¶

Entity Handlers for SQL Destinations¶

Entity Bindings¶

Entity Bindings for JSON¶

Entity Bindings for XML¶

Entity Bindings for SQL¶

Entity Bindings for URL¶

Data Change Notification¶

DCN Batches¶

DCN Examples¶

DCN Security¶

`NoStore` Term¶

`NoLoad` Term¶

`RefreshBy` Term¶

`PartitionBy` Term¶