Skip to content

Caching Data in the Cloud

The service generator tool accepts an OData CSDL XML file (for OData version 2.0 or 4.0) as input and generates a Java EE web application project that can build a deployable WAR file for the implementation of an OData service. This document describes how to annotate the OData CSDL XML file to configure a Cache Database.

Cache Databases

The basic Conventions for OData Metadata allow you to create an OData service that is implemented atop of a SQL database. In that case, the SQL database can be considered to be a Back-end System.

It is sometimes helpful to synchronize data between an existing back-end system and a Cache Database. The generated OData service acts as an intermediary between client applications and the back-end system, and uses the cache database to add capabilities that might not be provided by the back-end system, such as OData change tracking.

A cache database is particularly useful for the support of occasionally-connected mobile client applications. Data from the back-end system is periodically synchronized (pulled or pushed) into the cache database, from where it can be downloaded by client applications. Subsequently, client applications can use OData delta links to efficiently download only changed data. Data changes can later be uploaded to the back-end system via the generated OData service, which treats the cache database as a write-through cache.

Change Tracking

Cache databases rely upon the service generator's features for change tracking, which are separately documented.

Custom Annotations

The CSDL metadata for a cache database will include annotations using terms from one or more of the following annotation vocabularies:

  • The Cache vocabulary, which defines terms relating to the implementation of cache databases.

  • The HTTP vocabulary, which defines terms relating to the implementation of OData operations delegating to REST back-end systems using HTTP.

  • The JCo vocabulary, which defines terms relating to the implementation of OData operations delegating to ABAP back-end systems using JCo.

  • The SQL vocabulary, which defines terms relating to the implementation of OData operations delegating to SQL database back-end systems (as well as the cache database itself).

To enable a cache database:

  • Annotate the EntityContainer with SQL.CacheDatabase to specify that the SQL database managed by the OData service will be a cache database.

  • Annotate the EntityContainer with SQL.TrackChanges to enable change tracking.

  • Annotate the EntityContainer with SQL.TrackDownloads to enable download tracking.

  • Enable client registrations with a ClientRegistration entity type and ClientRegistrationSet entity set.

Example:

<edmx:Edmx Version="4.0" xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://docs.oasis-open.org/odata/ns/edmx http://docs.oasis-open.org/odata/odata/v4.0/os/schemas/edmx.xsd http://docs.oasis-open.org/odata/ns/edm http://docs.oasis-open.org/odata/odata/v4.0/os/schemas/edm.xsd">
    <edmx:Reference Uri="vocabularies/com.sap.cloud.server.odata.sql.v1.xml">
        <edmx:Include Namespace="com.sap.cloud.server.odata.sql.v1" Alias="SQL"/>
    </edmx:Reference>
    <edmx:DataServices>
        <Schema Namespace="my.schema" Alias="Self" xmlns="http://docs.oasis-open.org/odata/ns/edm">
            <!-- ClientRegistration entity type omitted for brevity. -->
            ...
            <EntityContainer Name="MyService">
                <Annotation Term="SQL.CacheDatabase"/>
                <Annotation Term="SQL.TrackChanges"/>
                <Annotation Term="SQL.TrackDownloads"/>
                <!-- ClientRegistrationSet entity set omitted for brevity. -->
                ...
            </EntityContainer>
        </Schema>
    </edmx:DataServices>
</edmx:Edmx>

Client Applications

Because cache databases are usually used in support of offline-enabled clients, it is recommended that you build client applications using the SAP Cloud Platform SDK for Android or iOS together with Offline OData (the OfflineODataProvider API), which communicates with the generated OData service via the Offline OData server-side component in SAP Cloud Platform Mobile Services. This combination enables optimized synchronization of changes between client applications and back-end systems.

When you use Offline OData with an OData service for a cache database, please take particular note of the following topics which are important to understand with regard to establishing communication between the client application and the cache database.

Cached Entities

Cached entities are entities that will contain copies of data from a back-end system. The purpose of cached entities is to remove the burden of having to implement OData change tracking from the back-end system, or to remove the burden of having to implement change tracking in a way that supports complex selection criteria (see Download Queries).

RefreshBy Term

You can apply the Cache.RefreshBy term to an entity type to indicate how entities in the cache are to be updated. The value of this term is a string, as a comma-separated list, which may include dcn (if the cache will be modified by Data Change Notification, and may also include either loadAll or loadPartition (see Cache.PartitionBy below).

Example:

<EntityType Name="Region">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Key>
        <PropertyRef Name="RegionID"/>
    </Key>
    <Property Name="RegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

Note

Any entity type without customized entity handlers and that does not use dcn will be stored directly in the cache database and will not be considered as a cached entity since it will be assumed that the entity type does not exist in any back-end system. Any such entity type can still be downloaded and uploaded by client applications.

PartitionBy Term

You can apply the Cache.PartitionBy term to an entity type to indicate that entities in the cache should be refreshed one partition at a time (using [loadPartition]). For example, perhaps the back-end system has no operation that can retrieve all Customer entities in one call, but instead can only retrieve the customer entities for a particular Region (entity type) by RegionID (key property). Then you could partition the customer entity type by Region/RegionID.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="Region/RegionID"/>
    <Key>
        <PropertyRef Name="CustomerID"/>
    </Key>
    <Property Name="CustomerID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="RegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

The source partitioning item (e.g. Region/RegionID) must have a corresponding property (e.g. RegionID) in the target entity type (e.g. Customer). If the corresponding target property has a different name, e.g. MyRegionID, then the partitioning item syntax should be prefixed with an assignment to the target property.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="MyRegionID = Region/RegionID"/>
    <Key>
        <PropertyRef Name="CustomerID"/>
    </Key>
    <Property Name="CustomerID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="MyRegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

Partitioning by Multiple Properties

If you need to uniquely identify a partition using multiple properties of the partitioning entity, you should provide them as a comma-separated list.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="Region/City, Region/State"/>
    <Key>
        <PropertyRef Name="CustomerID"/>
    </Key>
    <Property Name="CustomerID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    <Property Name="City" Type="Edm.String" Nullable="false" MaxLength="50"/>
    <Property Name="State" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

Partitioning by Client Registration

If you need to partition an entity type by client registration, use client as the value for the Cache.PartitionBy annotation.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="client"/>
    <Key>
        <PropertyRef Name="CustomerID"/>
    </Key>
    <Property Name="CustomerID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="RegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

You can also partition by client in combination with regular partitioning properties.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
    <Annotation Term="Cache.PartitionBy" String="client, Region/RegionID"/>
    <Key>
        <PropertyRef Name="CustomerID"/>
    </Key>
    <Property Name="CustomerID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="RegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

Note

Partitioning by client registration can result in multiple copies of back-end entities (i.e. one copy per registered client) being present in the cache database. This can be convenient if the back-end system has an existing operation that returns client-specific data. For data that is expected to be shared between clients (such as reference/master data), partitioning by client registration is not recommended as it can result in excessive utilization of storage space within the cache database.

When to Refresh

For an entity type that uses [loadAll] or [loadPartition] (in addition to or instead of dcn), one of the following annotations should be used to indicate how often (and when) the cache for that entity type will be refreshed.

  • The Cache.Schedule term is used to indicate that the cached entities for an entity type should be refreshed at a certain time of day (with UTC time, not local time), expressed as an Edm.TimeOfDay which uses the lexical representation of XML Schema time.

    Example:

    <EntityType Name="Region">
        <Annotation Term="Cache.RefreshBy" String="loadAll"/>
        <Annotation Term="Cache.Schedule" TimeOfDay="02:00:00"/>
        ...
    </EntityType>
    
  • The Cache.Timeout term is used to indicate that the cached entities for an entity type should be refreshed at a specified interval, expressed as an Edm.Duration which uses the lexical representation of XML Schema dayTimeDuration. If not specified, the default timeout is one hour (except if Cache.Schedule was specified, in which case the refresh is performed once daily).

    Example:

    <EntityType Name="Region">
        <Annotation Term="Cache.RefreshBy" String="loadAll"/>
        <Annotation Term="Cache.Timeout" Duration="PT4H"/>
        ...
    </EntityType>
    
  • The Cache.LoadAfter term is used to indicate that the cached entities for an entity type should be refreshed after the entities for some other entity type. This will often be the case for partitioned entity types, and also for child entity types in a parent/child relationship.

    Example:

    <EntityType Name="Customer">
        <Annotation Term="Cache.LoadAfter" String="Region.loadAll"/>
        <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
        <Annotation Term="Cache.PartitionBy" String="Region/RegionID"/>
        ...
    </EntityType>
    
  • The Cache.OnDemand term is used to indicate that the cached entities for an entity type should be refreshed right before a client's download query is executed, subject also to Cache.Timeout, which defaults to one hour if not specified. Shorter timeouts (such as PT0S to force cache refresh on each client download) should be used with care, since, if clients frequently execute download queries, the resulting load on the back-end system and/or cache database may be excessive.

    Example:

    <EntityType Name="CustomerVisit">
        <Annotation Term="Cache.OnDemand"/>
        <Annotation Term="Cache.RefreshBy" String="loadPartition"/>
        <Annotation Term="Cache.PartitionBy" String="client"/>
        ...
    </EntityType>
    
  • The Cache.OnStartup term is used to indicate that the cached entities for an entity type should be refreshed at server startup time. This can be convenient in a test system (combined with either Cache.Schedule or Cache.Timeout) so that testers do not need to wait for a specified time of day or elapsed time interval before testing that data is being refreshed correctly in the cache database.

    Example:

    <EntityType Name="Region">
        <Annotation Term="Cache.RefreshBy" String="loadAll"/>
        <Annotation Term="Cache.OnStartup"/>
        <Annotation Term="Cache.Timeout" Duration="PT1H"/>
        ...
    </EntityType>
    

When choosing between Cache.Schedule and Cache.Timeout, consider using Cache.Schedule (for daily refresh) to schedule cache refreshes for a time of day when not many clients are expected to be connected. This is particularly useful for entity sets that contain millions of entities. Consider using Cache.Timeout (e.g. for hourly refresh) when data must be more up-to-date for clients than daily scheduling would permit. However, note that timeout-based refresh may then occur during busy periods of client download activity. Because both cache refreshes and client downloads place a burden on the database server, configuring the timing of cache refreshes and client downloads to separate those activities may be helpful to overall system performance. If cache refreshes and client downloads are not separated, then the database server may need additional resources (such as CPU and RAM) to ensure that the impact on client download performance is acceptable to users.

An alternative to Cache.Schedule and Cache.Timeout (or even supplementing them) is to use DCN to refresh data in the cache database in response to change notifications from the back-end system. DCN usually reduces the workload of both the cache database and the back-end system, as compared with relying on schedules or timeouts. This is because schedules and timeouts are pull-based solutions that aren't aware of whether back-end system data has actually changed, whereas DCN is a push-based solution, which can be responsive to actual back-end changes.

Entity Handlers

Customizing the Generated Service describes how you can use handler classes to customize the implementation of OData CRUD operations (create / read / update / delete) for OData entity types.

When you implement a cache database, you can implement the following handler methods for each entity type to interact with the back-end system.

  • loadAll, which should query the back-end system for all entities of a particular type and return those entities so that the caching system can merge any new or changed entities into the cache database (and delete any entities which are no longer present in the back-end system from the cache database).

  • loadPartition, which should query the back-end system for some entities of a particular type (as indicated by the Cache.PartitionBy term), and return those entities so that the caching system can merge any new or changed entities into the cache database (and delete any entities which are no longer present in the back-end system from the cache database).

  • createEntity, which should create an entity in the back-end system, obtain the (possibly generated) key properties and (possibly altered) non-key properties, and then create a corresponding entity in the cache database.

  • updateEntity, which should update an entity in the back-end system, obtain the (possibly altered) non-key properties, and then update the corresponding entity in the cache database.

  • deleteEntity, which should delete an entity from the back-end system, and then delete the corresponding entity from the cache database.

You can customize Handler classes to interact with arbitrary back-end systems, or in certain cases (for back ends accessible via HTTP, JCo or JDBC) they can be generated automatically using the annotation terms HTTP.Request, JCO.Function or SQL.Statement (see below for term descriptions).

OData Back-End Systems

If the back-end system being cached is itself an OData service, then entity handlers can be automatically generated by using a single Cache.ODataBackend annotation within the EntityContainer, together with the appropriate annotations for cached entities such as Cache.RefreshBy, Cache.PartitionBy etc.

Example:

<edmx:Edmx Version="4.0"
    xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://docs.oasis-open.org/odata/ns/edmx http://docs.oasis-open.org/odata/odata/v4.0/os/schemas/edmx.xsd http://docs.oasis-open.org/odata/ns/edm http://docs.oasis-open.org/odata/odata/v4.0/os/schemas/edm.xsd">
    <edmx:Reference Uri="vocabularies/com.sap.cloud.server.odata.sql.v1.xml">
        <edmx:Include Namespace="com.sap.cloud.server.odata.sql.v1" Alias="SQL"/>
    </edmx:Reference>
    <edmx:DataServices>
        <Schema Namespace="my.schema" Alias="Self" xmlns="http://docs.oasis-open.org/odata/ns/edm">
            ...
            <EntityContainer Name="MyService">
                <Annotation Term="Cache.ODataBackend" String="destination-name"/>
                <Annotation Term="SQL.CacheDatabase"/>
                <Annotation Term="SQL.TrackChanges"/>
                <Annotation Term="SQL.TrackDownloads"/>
                ...
            </EntityContainer>
        </Schema>
    </edmx:DataServices>
</edmx:Edmx>

The Cache.ODataBackend term specifies an HTTP destination name for the OData back-end system. See below how to define an HTTP destination in various target deployment environments.

It is not mandatory to use this technique when caching an OData back-end system. Manually coded entity handlers or HTTP.Request annotations can be used instead. However using the Cache.ODataBackend term is likely to require considerably less developer effort.

Entity Handlers for HTTP

The HTTP.Destination term is used to specify the destination name of a back-end system that is accessed using HTTP or HTTPS. Such a system would normally be a RESTful Web Service using JSON or XML for request and response payloads.

Example:

<EntityType Name="Region">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Annotation Term="HTTP.Destination" String="destination-name"/>
    <Key>
        <PropertyRef Name="RegionID"/>
    </Key>
    <Property Name="RegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

Depending on the target deployment environment, the connectivity information for the named back-end service is defined as follows:

  • Apache TomEE (or Tomcat): file <tomee-home-directory>/conf/http/<destination-name>.properties, containing a url property with the service root URL, and optionally username and password properties for HTTP Basic authentication.

  • Eclipse Virgo (or SMP 3.1+): file <virgo-home-directory>/configuration/http/<destination-name>.properties, containing a url property with the service root URL, and optionally username and password properties for HTTP Basic authentication.

  • SAP Cloud Platform (Cloud Foundry or Neo environment): use the SAP Cloud Platform Cockpit to define an HTTP destination for the application.

If username or password properties are provided for preemptive basic authentication, then either or both may use client registration bindings to obtain the necessary credential from a property of the ClientCredentials entity. Any more complex requirements for HTTP back-end authentication can be implemented by customizing the generated HeaderProvider class.

The HTTP.Request term is used to specify a String providing the method and URL for an HTTP request (see entity bindings for URL), and a Qualifier specifying the name of an entity handler method (loadAll, loadPartition, createEntity, updateEntity, or deleteEntity), as well as nested annotations (HTTP.RequestHeaders, HTTP.RequestBody, HTTP.ResponseHeaders, HTTP.ResponseBody) as required for entity bindings using JSON or XML that match headers and content from requests and responses with entity properties.

The HTTP Content-Type and Accept headers can be determined automatically from the entity bindings, if the entity bindings use JSON or XML and the corresponding expected content type is application/json or application/xml. So, using the HTTP.RequestHeaders term is often not required.

Note

Important: a POST (create) operation for an OData entity is required to produce a Location header in the response, so if the primary key of the created entity is not provided by the client application but rather is generated by the back-end system, a response binding is required for the createEntity method to bind the back-end-generated key to the entity's key property.

Example:

<EntityType Name="Region">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Annotation Term="HTTP.Destination" String="destination-name"/>
    <Annotation Term="HTTP.Request" Qualifier="loadAll" String="GET /regions">
        <Annotation Term="HTTP.ResponseBody">
            <String>
                [
                    {
                        "id": "${entity.RegionID}",
                        "name": "${entity.Name}"
                    }
                ]
            </String>
        </Annotation>
    </Annotation>
    <Annotation Term="HTTP.Request" Qualifier="createEntity" String="POST /regions">
        <Annotation Term="HTTP.RequestHeaders">
            <String>
                {
                    "Accept": "application/json"
                }
            </String>
        </Annotation>
        <Annotation Term="HTTP.RequestBody">
            <String>
                {
                    "name": "${entity.Name}"
                }
            </String>
        </Annotation>
        <Annotation Term="HTTP.ResponseHeaders">
            <!-- Note: it is unusual to bind response headers to entity properties -->
            <String>
                {
                    "X-Info": "${entity.Info}"
                }
            </String>
        </Annotation>
        <Annotation Term="HTTP.ResponseBody">
            <!-- Must bind the back-end-generated key to the entity's key property -->
            <String>
                {
                    "id": "${entity.RegionID}"
                }
            </String>
        </Annotation>
    </Annotation>
    <Annotation Term="HTTP.Request" Qualifier="updateEntity" String="PUT /regions/${entity.RegionID}">
        <Annotation Term="HTTP.RequestBody">
            <String>
                {
                    "name": "${entity.Name}"
                }
            </String>
        </Annotation>
    </Annotation>
    <Annotation Term="HTTP.Request" Qualifier="deleteEntity" String="DELETE /regions/${entity.RegionID}"/>
    <Key>
        <PropertyRef Name="RegionID"/>
    </Key>
    <Property Name="RegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    <Property Name="Info" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

Entity Handlers for SAP JCo

This section describes annotation terms for automatic generation of entity handlers that access a back-end system using the SAP Java Connector.

The JCO.Destination term is used to specify the destination name of the JCo back-end system.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Annotation Term="JCO.Destination" String="destination-name"/>
    ...
</EntityType>

Depending on the target deployment environment, the connectivity information for the named back-end service is defined as follows:

  • Apache TomEE (or Tomcat): file <tomee-home-directory>/conf/jco/<destination-name>.properties, containing the JCo connection properties. Please also ensure that sapjco3.* from SAP Java Connector 3.0.19+ is installed into the server's lib directory.

  • Eclipse Virgo (or SMP 3.1+): file <virgo-home-directory>/conf/jco/<destination-name>.properties, containing the JCo connection properties. Please also ensure that sapjco3.* from SAP Java Connector 3.0.19+ is installed into the server's lib directory, and edit the configuration/config.ini file and add (or edit) the org.osgi.framework.system.packages.extra property to specify the Java package name(s) for the JCo runtime, e.g. com.sap.conn.jco;version=3.0.19.

  • SAP Cloud Platform (Cloud Foundry or Neo environment) - use the SAP Cloud Platform Cockpit to define a JCo destination for the application.

The JCO.Function term specifies a String providing the name of a JCo function, and a Qualifier specifying the name of a handler method (loadAll, loadPartition, createEntity, updateEntity, or deleteEntity), as well as nested annotations (JCO.InputRecord, JCO.OutputRecord) as required for entity bindings using JSON that match JCo input and output parameters from requests and responses with entity properties.

Example:

<EntityType Name="Customer">
    <Annotation Term="Cache.OnStartup"/>
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Annotation Term="Cache.Timeout" Duration="PT1H"/>
    <Annotation Term="JCO.Destination" String="destination-name"/>
    <Annotation Term="JCO.Function" Qualifier="loadAll" String="BAPI_FLCUST_GETLIST">
        <Annotation Term="JCO.OutputRecord">
            <!-- Note: a JSON array binding is used for a JCO table parameter -->
            <String>
                {
                    "CUSTOMER_LIST":
                    [
                        {
                            "CITY": "${entity.City}",
                            "COUNTR": "${entity.Country}",
                            "COUNTR_ISO": "${entity.CountryISO}",
                            "CUSTNAME": "${entity.Name}",
                            "CUSTOMERID": "${entity.CustomerID}",
                            "EMAIL": "${entity.Email}",
                            "FORM": "${entity.Honorific}",
                            "PHONE": "${entity.Phone}",
                            "POBOX": "${entity.POBox}",
                            "POSTCODE": "${entity.PostCode}",
                            "REGION": "${entity.Region}",
                            "STREET": "${entity.Street}"
                        }
                    ]
                }
            </String>
        </Annotation>
    </Annotation>
    <Annotation Term="JCO.Function" Qualifier="createEntity" String="BAPI_FLCUST_CREATEFROMDATA">
        <Annotation Term="JCO.InputRecord">
            <!-- Note: a JSON object binding is used for a JCO structure parameter -->
            <String>
                {
                    "CUSTOMER_DATA":
                    {
                        "CITY": "${entity.City}",
                        "COUNTR": "${entity.Country}",
                        "COUNTR_ISO": "${entity.CountryISO}",
                        "CUSTNAME": "${entity.Name}",
                        "CUSTTYPE": "B",
                        "EMAIL": "${entity.Email}",
                        "FORM": "${entity.Honorific}",
                        "PHONE": "${entity.Phone}",
                        "POBOX": "${entity.POBox}",
                        "POSTCODE": "${entity.PostCode}",
                        "REGION": "${entity.Region}",
                        "STREET": "${entity.Street}"
                    }
                }
            </String>
        </Annotation>
        <Annotation Term="JCO.OutputRecord">
            <!-- Note: a JSON field binding is used for a JCo primitive parameter -->
            <String>
                {
                    "CUSTOMERNUMBER": "${entity.CustomerID}"
                }
            </String>
        </Annotation>
    </Annotation>
    <Key>
        <PropertyRef Name="CustomerID"/>
    </Key>
    <Property Name="City" Type="Edm.String" Nullable="true" MaxLength="100"/>
    <Property Name="Country" Type="Edm.String" Nullable="true" MaxLength="10"/>
    <Property Name="CountryISO" Type="Edm.String" Nullable="true" MaxLength="10"/>
    <Property Name="CustomerID" Type="Edm.String" Nullable="false" MaxLength="8"/>
    <Property Name="Email" Type="Edm.String" Nullable="true" MaxLength="100"/>
    <Property Name="Honorific" Type="Edm.String" Nullable="true" MaxLength="20"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="100"/>
    <Property Name="Phone" Type="Edm.String" Nullable="true" MaxLength="50"/>
    <Property Name="POBox" Type="Edm.String" Nullable="true" MaxLength="20"/>
    <Property Name="PostCode" Type="Edm.String" Nullable="true" MaxLength="20"/>
    <Property Name="Region" Type="Edm.String" Nullable="true" MaxLength="100"/>
    <Property Name="Street" Type="Edm.String" Nullable="true" MaxLength="100"/>
</EntityType>

Entity Handlers for JDBC

This section describes annotation terms for automatic generation of entity handlers that access a back-end system using JDBC.

The SQL.DataSource term is used to specify the data source name for a back-end database system that is accessed using JDBC.

Example:

<EntityType Name="Region">
    <Annotation Term="Cache.RefreshBy" String="loadAll"/>
    <Annotation Term="SQL.DataSource" String="data-source-name"/>
    <Key>
        <PropertyRef Name="RegionID"/>
    </Key>
    <Property Name="RegionID" Type="Edm.Int64" Nullable="false"/>
    <Property Name="Name" Type="Edm.String" Nullable="false" MaxLength="50"/>
    ...
</EntityType>

Depending on the target deployment environment, the connectivity information for the JDBC data source is defined as follows:

  • Apache TomEE (or Tomcat): file <tomee-home-directory>/conf/jdbc/<data-source-name>.properties, containing data source configuration parameters as defined by Apache Commons DBCP. For production systems, it is recommended to use the testOnBorrow and validationQuery properties to ensure that the OData service will recover gracefully from broken database connections, and the maxTotal property to configure the maximum number of concurrent connections to the database should be permitted.

  • Eclipse Virgo (or SMP 3.1+): file <virgo-home-directory>/configuration/jdbc/<data-source-name>.properties, containing data source configuration parameters as defined by Apache Commons DBCP. For production systems, it is recommended to use the testOnBorrow and validationQuery properties to ensure that the OData service will recover gracefully from broken database connections, and the maxTotal property to configure the maximum number of concurrent connections to the database should be permitted.

  • SAP Cloud Platform (Cloud Foundry and Neo environments): See topic Using the TCP Protocol for Cloud Applications in the SAP Cloud Platform Connectivity documentation. This will only work for database products supporting JDBC via a SOCKS5 proxy, e.g. SAP HANA. If this limitation prevents connectivity to the back-end database, then a recommended solution is to deploy the generated OData service to an on-premise Apache TomEE environment, and use SAP Cloud Platform Connector to access the on-premise TomEE from SAP Cloud Platform Mobile Services.

Entity Bindings

A back-end operation may expect input parameters or return output parameters that need to be bound to entity properties when an entity handler method (loadAll, loadPartition, createEntity, updateEntity, or deleteEntity) is called. Operation parameters are matched with entity properties using entity bindings.

A regular entity binding is a string of the form ${entity.PropertyName}. It can be used for binding input or output parameters to the properties of the entity type within which the binding occurs.

An old entity binding is a string of the form ${old.PropertyName}. It can be used for binding input parameters to the old properties of the entity type within which the binding occurs. This is applicable to updateEntity and deleteEntity, and may be helpful for the avoidance of lost updates using optimistic concurrency control techniques.

A partition entity binding is a string of the form ${partition.PropertyName}. It can be used for binding input parameters to the properties of the partition type for an entity type which defines a [loadPartition] method, which will associate the input parameters with the property values corresponding to the partition which is currently being loaded into the cache database.

A client credentials binding is a string of the form ${client.PropertyName}. It can be used for binding input parameters to the properties of the ClientCredentials entity type, which will associate the input parameters with the property values corresponding to the client which is currently executing the request that is being delegated to the entity handler.

A client registration binding is a string of the form ${client.PropertyName}. It can be used for binding input parameters to the properties of the ClientRegistration entity type, which will associate the input parameters with the property values corresponding to the client which is currently executing the request that is being delegated to the entity handler.

A header structure binding is a string of the form ${header.StructureName.PropertyName}. It can be used to obtain parameter values from an HTTP header. Custom HTTP headers may be used for passing parameters other than entity properties. For example, within an entity type Customer, the binding ${header.CreateParameters.Reason} would expect an HTTP header X-Create-Parameters whose value is a data URI containing the Base64-encoded JSON representation of a CustomerCreateParameters complex value containing a Reason property. To facilitate this encoding of credentials, see ToJSON.dataURI and HttpHeaders.withData in the client SDK documentation, or refer to the following example showing the encoding steps.

Example encoding of parameters object with Reason of "special":

  • JSON-encoded object: {"Reason":"special"}
  • Base64-encoded JSON: eyJSZWFzb24iOiJzcGVjaWFsIn0=
  • Encoded as data URI: data:application/json;base64,eyJSZWFzb24iOiJzcGVjaWFsIn0=
  • HTTP header: X-Create-Parameters: data:application/json;base64,eyJSZWFzb24iOiJzcGVjaWFsIn0=

Note

Header structure bindings are provided to support interoperation with back-end systems whose operations may expect some parameters that do not exist as properties in the back-end entity model. It is preferable to use regular entity bindings wherever possible.

Entity Bindings for JSON

JSON entity bindings use a String containing the text for a JSON array or JSON object, possibly containing nested JSON arrays or objects, to arbitrary levels of nesting as required by the back-end operation. The ${entity.PropertyName} syntax is used within field values inside JSON objects.

Since JSON bindings are contained within a CSDL XML document, it is recommended to use CDATA sections so that XML special characters (less-than "<", greater-than ">", ampersand "&"), which may occur within the JSON bindings, do not need to be escaped using XML entity references.

Example:

<Annotation Term="HTTP.Request" Qualifier="updateEntity" String="PUT /patient/${entity.PatientID}">
    <Annotation Term="HTTP.RequestBody">
        <String><![CDATA[
            {
                "name": "${entity.Name}",
                "address": "${entity.Address}",
                "dateOfBirth": "${entity.DOB}",
                "special_<&>": "${entity.SpecialInfo}"
            }
        ]]></String>
    </Annotation>
</Annotation>

For binding results of a [loadAll] or [loadPartition] call, if the JSON bindings include multiple arrays, then one of those arrays needs to be designated as the one whose occurrences determine the number of result entities. By default the innermost nested array will implicitly designate the number of result entities, but it can be explicitly designated using a special first item within the array containing the string "for (entity of result)". This syntax is modeled after JavaScript Array comprehensions but is quoted within a string to be JSON-compatible.

Example:

<Annotation Term="HTTP.Request" Qualifier="loadAll" String="GET /order">
    <Annotation Term="HTTP.ResponseBody">
        <String>
            {
                "results":
                [
                    {
                        "orderId": "${entity.OrderID}"
                        [
                            "for (entity of result)",
                            {
                                "itemId": "${entity.ItemID}",
                                "quantity": ${entity.Quantity}"
                            }
                        ]
                    }
                ]
            }
        </String>
    </Annotation>
</Annotation>

Two additional special forms of JSON entity binding may be useful for processing the responses from some RESTful APIs.

  • An object field can be bound to @nextLink if the back-end system breaks query responses into separate linked documents.

  • An entity binding can be placed after a parenthesized group within a regular expression to extract only a portion of a JSON field value into an entity property.

Example:

<Annotation Term="HTTP.Request" Qualifier="loadAll" String="GET /region">
    <Annotation Term="HTTP.ResponseBody">
        <!-- Back-end system returns a series of documents with next links -->
        <!-- Back-end system returns URLs (e.g. "http://example.com/region/123/") rather than simple IDs -->
        <String><![CDATA[
            {
                "next": "@nextLink",
                "results":
                [
                    {
                        "url": ".*/(\\d+)${entity.RegionID}/",
                        "name": "${entity.Name}"
                    }
                ]
            }
        ]]></String>
    </Annotation>
</Annotation>

Entity Bindings for XML

XML entity bindings use a String containing the text for an XML document, possibly containing nested XML elements or attributes, to arbitrary levels of nesting, as required by the back-end operation. The ${entity.PropertyName} syntax is used within the text of XML elements and attributes.

Since XML bindings are contained within a CSDL XML document, it is recommended to use CDATA sections so that XML special characters (less-than "<", greater-than ">", ampersand "&"), which may occur within the XML bindings, do not need to be escaped using XML entity references.

Example:

<Annotation Term="HTTP.Request" Qualifier="updateEntity" String="PUT /patients">
    <Annotation Term="HTTP.RequestBody">
        <String><![CDATA[
            <patient id="${entity.PatientID}">
                <name>${entity.Name}</name>
                <address>${entity.Address}</address>
                <dateOfBirth>${entity.DOB}</dateOfBirth>
            <patient>
        ]]></String>
    </Annotation>
</Annotation>

For binding results of a [loadAll] or [loadPartition] call, one of the non-root elements should be designated as the one whose occurrences determine the number of result entities, using a special XML attribute: for-each="entity of result".

Example:

<Annotation Term="HTTP.Request" Qualifier="loadAll" String="GET /patients">
    <Annotation Term="HTTP.ResponseBody">
        <String><![CDATA[
            <patients>
                <patient id="${entity.PatientID}" for-each="entity of result">
                    <name>${entity.Name}</name>
                    <address>${entity.Address}</address>
                    <dateOfBirth>${entity.DOB}</dateOfBirth>
                <patient>
            </patients>
        ]]></String>
    </Annotation>
</Annotation>

Two additional special forms of XML entity binding may be useful for processing the responses from some RESTful APIs.

  • An XML element or attribute can be bound to @nextLink if the back-end system breaks query responses into separate linked documents.

    Note

    This form of binding is not yet supported by the csdl-to-war tool.

  • An entity binding can be placed after a parenthesized group within a regular expression to extract only a portion of an XML element or attribute into an entity property.

    Example:

    <Annotation Term="HTTP.Request" Qualifier="loadAll" String="GET /region">
        <Annotation Term="HTTP.ResponseBody">
            <!-- Back-end system returns a series of documents with next links -->
            <!-- Back-end system returns URLs (e.g. "http://example.com/region/123/") rather than simple IDs -->
            <String><![CDATA[
                <mydoc next="@nextLink">
                    <region for-each="entity of result">
                        <url>.*/(\\d+)${entity.RegionID}/</url>
                        <name>${entity.Name}</name>
                    </region>
                </mydoc>
            ]]></String>
        </Annotation>
    </Annotation>
    

Entity Bindings for JDBC

JDBC entity bindings use a String containing the text for a SQL statement, as expected by the target database, expressed as Embedded SQL using :PropertyName host variable syntax instead of ${entity.PropertyName} for regular entity bindings, using :old.PropertyName for old entity bindings, using :partition.PropertyName for partition entity bindings, and using :client.PropertyName for client property bindings.

Because JDBC bindings are contained within a CSDL XML document, it is recommended to use CDATA sections so that XML special characters (less-than "<", greater-than ">", ampersand "&"), which may occur within the JDBC bindings, do not need to be escaped using XML entity references.

Example:

<Annotation Term="SQL.Statement" Qualifier="loadAll">
    <String><![CDATA[
        select id, name, address, date_of_birth
        into :PatientID, :Name, :Address, :DOB
        from patient
    ]]></String>
</Annotation>

<Annotation Term="SQL.Statement" Qualifier="createEntity">
    <String><![CDATA[
        insert into patient (name, address, date_of_birth)
        values (:Name, :Address, :DOB)
        returning id
    ]]></String>
</Annotation>

<Annotation Term="SQL.Statement" Qualifier="updateEntity">
    <String><![CDATA[
        update patient
        set name = :Name, address = :Address, date_of_birth = :DOB
        where id = :PatientID
    ]]></String>
</Annotation>

<Annotation Term="SQL.Statement" Qualifier="deleteEntity">
    <String><![CDATA[
        delete from patient where id = :PatientID
    ]]></String>
</Annotation>

The provided SQL statement will not be passed directly to the back-end database. It will be preprocessed for use with a JDBC PreparedStatement:

  • All host variable references (e.g. :Name) will be replaced with a JDBC PreparedStatement parameter placeholder (?).

  • The into clause for select statements is provided to match result columns with entity properties. It will not be passed to the database.

  • Unless the client application is expected to provide the primary key, the returning clause for insert statements is required to indicate that a database-generated key will be returned. It will be transformed into the use of a database-specific mechanism (usually PreparedStatement.getGeneratedKeys()) for retrieving the generated key. For databases supporting key generation with identity columns, the key column does not need to be included in the values clause. For databases supporting key generation with sequence types, the values clause may need to include the database-specific syntax for obtaining the next value from the sequence.

    Example:

    <Annotation Term="SQL.Statement" Qualifier="createEntity">
        <String><![CDATA[
            insert into patient (id, name, address, date_of_birth)
            values (patient_id_seq.nextval, :Name, :Address, :DOB)
            returning id
        ]]></String>
    </Annotation>
    

Entity Bindings for URL

URL entity bindings may appear within the String for an HTTP.Request annotation. The ${entity.PropertyName}, ${partition.PropertyName} and ${client.PropertyName} property binding forms can all be used. Additionally, an entity binding can be wrapped with encode(...) to percent-encode strings which may contain reserved URL characters.

Example:

<Annotation Term="HTTP.Request" Qualifier="updateEntity" String="PUT /customer/${entity.CustomerID}">
    ...
</Annotation>

<Annotation Term="HTTP.Request" Qualifier="updateEntity" String="PUT /customer/${encode(entity.Name)}">
    ...
</Annotation>

Data Change Notification

Previously it was described how to use a SQL.RefreshBy annotation with [loadAll] or [loadPartition] so that data will be pulled periodically from the back-end system into the cache database.

Another option, which can be more efficient if the back-end system has some form of built-in change logging, is to enhance the back-end system so that data changes will be pushed periodically into the cache database via the OData service. This is known as Data Change Notification (DCN).

To minimize unnecessary processing in the OData service, the back-end system should make some attempt to send only changed data in DCN messages, i.e. data that was changed in the back-end system after a previously successful DCN request.

DCN Batches

In order to reduce network latency, and considering that it is typical for multiple data changes from the back-end system to be pushed into the cache database around the same time, DCN requests must be sent in batches. A batch with only one change is acceptable, but batch sizes of 100 or more are recommended (unless the batches contain large data, such as images, in which case a smaller batch size, such as 10, is recommended).

A DCN batch must be sent to the URL dcn/$batch relative to the service root (a regular OData batch request would be sent to the URL $batch relative to the service root). Two DCN batch formats are supported:

  • OData JSON batch format using a method of put, patch or delete. The DCN sender should format the entities within the payload using OData 4.0 JSON representation, even if the service metadata uses a prior version.

  • SAP Mobile Platform MBO DCN format with payload, and an operation (op) of :upsert or :delete. This format is intended only for backwards compatibility with the SAP Mobile Platform MBO Runtime. New development should use the OData JSON batch format.

Note the following:

  • For efficiency, each DCN batch will be executed within a single database transaction (even for OData JSON batch format without atomicityGroup being specified).

  • Back-end systems initiating DCN requests may wish to have multiple processes or threads simultaneously processing non-overlapping subsets of the changed data to take advantage of the available capacity of multiprocessor systems. However, if DCN requests are likely to overlap in time with periods where clients are downloading changes, excessive parallelism of DCN requests may degrade client download performance. In all cases when using DCN, some benchmarking should be done to determine appropriate batch sizes and levels of parallelism.

  • OData JSON batch format expects the url field to include an entity set name (e.g. CustomerSet), whereas SAP Mobile Platform MBO DCN format expects the mbo field to include an entity type name (e.g. Customer).

  • DCN batches with OData JSON batch format cannot use the post method, because the back-end system will not usually be certain whether or not the cache database already contains a particular entity that has been changed. Allowing post could potentially result in many create conflicts (due to duplicate keys), likely causing poor system throughput. Thus the put method is used to implement upsert behavior.

  • DCN batches with OData JSON batch format can use the patch method if the back-end system knows it has previously put an entity and now wants to change a subset of its properties. However the implementation of patch will in most cases be slower than put, even though put must provide all properties, because put can make use of database-level batched updates, whereas patch uses a non-batched select-then-update strategy. Patching an entity which does not exist in the cache will result in status 404 (Not Found) for the corresponding item in the response payload.

  • Other than status of 404 (Not Found) for patching non-existing entities, each item in the response payload will indicate status of 204 (No Content) when using OData batch format, and success of true when using MBO DCN format.

  • If the enclosing HTTP POST response has a status other than 200 (OK), then the back-end system should assume that the DCN processing might not have completed successfully (due to database or network failure, for instance), in which case it should be repeated (POSTed again) after a suitable delay (see exponential backoff). If such failures are consistent rather than intermittent, it might indicate that the request payload is invalid (possibly containing invalid entity type, set, or property names), in which case the back-end system code issuing the DCN requests will need to be corrected. The embedded metrics service provides a dcnBatchFailures metric to facilitate checking for DCN failures.

DCN Examples

HTTP request message using OData JSON batch format:

POST /dcn/$batch HTTP/1.1
Accept: application/json
OData-Version: 4.0
Content-Type: application/json
Content-Length: ...

{
    "requests":
    [
        {"id": "1", "method": "put", "url": "Customers(123)",
            "body": {"Name": "Jean Williams", "Address": "33 Main St."}},
        {"id": "2", "method": "patch", "url": "Customers(456)",
            "body": {"Address": "25 Oak St."}},
        {"id": "3", "method": "delete", "url": "Customers(789)"}
    ]
}

HTTP response message using OData JSON batch format:

HTTP/1.1 200 OK
OData-Version: 4.0
Content-Type: application/json
Content-Length: ...

{
    "responses":
    [
        {"id": "1", "status": 204},
        {"id": "2", "status": 204},
        {"id": "3", "status": 204}
    ]
}

HTTP request message using SAP Mobile Platform MBO DCN format:

POST /dcn/$batch HTTP/1.1
Content-Type: application/json
Accept: application/json
Content-Length: ...

{
    "messages":
    [
        {"id": "1", "op": ":upsert", "mbo": "Customer",
            "cols": {"CustomerID": 123, "Name": "Jean Williams", "Address": "33 Main St."}},
        {"id": "2", "op": ":delete", "mbo": "Customer",
            "cols": {"CustomerID": 789}}
    ]
}

HTTP response message using SAP Mobile Platform MBO DCN format:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: ...

[
    {"recordID": "1", "success": true, "statusMessage": ""},
    {"recordID": "2", "success": true, "statusMessage": ""}
]

DCN Security

If the generated OData service is deployed with security enabled, then all DCN requests must be sent by an authenticated service user possessing the DCNRole security role.


Last update: September 29, 2020