Change Tracking for CAP Services¶

The tools provided by the SAP Cloud Application Programming Model make it easy to model, generate and deploy an OData service.

If the generated OData service is to be consumed by occasionally-connected mobile clients, together with CloudSyncProvider or OfflineODataProvider in one of the SAP Business Technology Platform client SDKs, then it is highly beneficial for data synchronization performance to augment the CAP service with OData change tracking capabilities.

In particular, the performance of delta downloads, where the client wishes to download only the back-end data changes since the previous download, can be expected to be achieved in seconds if change tracking is enabled, compared with minutes to hours if change tracking is not enabled (assuming there was a high volume of data in the initial download).

Requirements¶

The CAP service is already deployed to the SAP Business Technology Platform Cloud Foundry environment, using either the Node.js runtime or the Java runtime.
The CAP service is already configured to store CAP entities in a SAP HANA Cloud schema. CAP entities that proxy/wrap non-CAP back-end APIs and aren't presently stored in the HANA schema can also be supported for change tracking.

Architecture¶

The mobile back-end tools is used to generate and deploy a delta service, which is a proxy service that is positioned between mobile clients and the CAP service. Change tracking is enabled by either delta views or cache tables.

For CAP entities (including projections) that are mapped by the CAP tools to simple database views, the mobile back-end tools will generate tracking tables, tracking triggers and delta views (in the same SAP HANA Cloud schema used by the existing CAP service) for change tracking and the delta service will use the delta views to enable efficient OData change tracking.
For CAP entities that are mapped by the CAP tools to complex database views (or to external entities, not present in the HANA schema), the mobile back-end tools generates cache tables (in the same SAP HANA Cloud schema used by the existing CAP service) and the delta service uses the cache tables to enable efficient OData change tracking.

Effectively, the mobile back-end tools will use a hybrid of Working with Existing Database Tables and Caching Data in the Cloud (OData Back-End Systems) to enable change tracking for each CAP entity as efficiently as possible.

The generated delta service uses a Java runtime to ensure optimum performance, and should be deployed to run in Apache Tomcat or TomEE in the SAP BTP Cloud Foundry environment, or otherwise in an on-premise environment.

Development¶

See Installing the Tools for initial setup instructions including the Java Development Kit, Apache Maven and Cloud Foundry CLI. Presently only Visual Studio Code is supported for generation and execution of the necessary tasks. SAP Business Application Studio will be supported in a later release (currently it does not support task dependencies in tasks.json).

Creating Tasks¶

Task creation only needs to be done once per CAP project, unless the provided information is subsequently changed.

In VS Code, open the CAP project's root folder as a workspace using File > Open Workspace From File..., or if there isn't an existing workspace file, then use File > Open Folder... and then File > Save Workspace As... to create a workspace file.
Use the Command Palette to invoke the MBT: Create delta-enabled CAP service tasks.json file command. This will prompt you for a variety of information about the CAP project, including:
- The CAP project folder.
- The CAP application name, e.g. bookshop-srv.
- The CAP service model, e.g. srv/my-service.cds.
- The HANA service name, e.g. bookshop-db.
- The MBT application name, e.g. bookshop-delta.
- The CAP service's URL, e.g. https://xxx.cfapps.yyy.hana.ondemand.com/api/MyService.
Use File > Add Folder to Workspace... to add the srv/delta folder to the workspace. This ensures that the generated tasks will be visible in the Terminal menu.
To customize the generated tasks.json, see Visual Studio Code Tasks.

Note: the generated tasks.json includes the following additional options for the csdl-to-war command, which are not elsewhere documented. These additional options enable change tracking for the CAP service.

"-cap:service",
- Specifies that the OData back-end system is a CAP service.
"-track:changes",
- Equivalent to specifying the OData SQL.TrackChanges annotation.
"-track:downloads",
- Equivalent to specifying the OData SQL.TrackDownloads annotation.
"-delta:triggers",
- Enables the delta views strategy where applicable.
"-proxy:cache", "<cap-application-name>",
- Enables proxying of non-delta requests (e.g. cache loads, entity updates) to the CAP service.
- Enables the cache tables strategy where applicable.
"-schema:cds", "../../srv/<service-model>.cds"
- Specifies the location of the CAP service model file.
"-hdi:deploy", "../../db/src/delta",
- Specifies the location for generation of extra HDI artifacts to support change tracking.

Executing Tasks¶

Always ensure that the project is opened using the File > Open Workspace From File... menu option. Opening the CAP project folder using Open Folder... can result in task execution failures.

A task can be executed by selecting the task name from the dialog opened by the Terminal > Run Task... menu option.

If a task fails due to Cloud Foundry CLI session timeout, use the cf login command from a terminal opened by Terminal > New Terminal to provide your credentials, then re-execute the failed task.

The following primary tasks are defined for managing the delta service.

To generate and deploy the delta service, choose the build-and-deploy task. This task can and should be repeated any time a change is made to the CAP data model or the CAP service model.
- Note: The build-and-deploy task can fail when it uses cds deploy to deploy HANA artifacts if the HANA instance rejects the IP address of the computer running the task. If this happens, please refer to Deploy using cds deploy (you can add the "--tunnel-address", "<host:port>", arguments to the cds deploy command in task cds-to-hana in your tasks.json file), or alternatively Allow/Deny IP Addresses for incoming connections.
To download logs from the running delta service, choose the download-logs task.
- For more details on monitoring options, see logging and metrics.
To restart the delta service, choose the restart-server task.

The following tasks are useful once you are ready to integrate with SAP Mobile Services, in preparation for the creation of a client application.

setup-mobile-app (mobile clients should access the delta service via SAP Mobile Services)
setup-mbt-license (see SAP Mobile Services License)

The remaining tasks are dependent tasks, which support the above-listed tasks.

Deployment Artifacts¶

To aid understanding of the delta service implementation, please take note of the following generated artifacts.

The db/src/delta folder, containing the generated HANA deployment (HDI) artifacts for delta views and cache tables. These files supplement the HDI artifacts that were generated by the CAP tools, ensuring that anything needed in the HANA schema to support change tracking will be automatically added during execution of the build-and-deploy task.
The srv/delta folder, a sub-project folder containing the main files for the delta service.
The srv/delta/.vscode/tasks.json file, containing the generated tasks.
The srv/delta/logs folder, containing delta service logs downloaded by the download-logs task.
The srv/delta/src folder, containing generated Java source files for the delta service implementation. Customization of the generated Java files is permitted, although it is generally recommended to customize the CAP service implementation files rather than the delta service implementation files.
Some OData metadata files. Do not edit any of these files, as your changes will be lost upon any subsequent execution of the build-and-deploy task.
- The file srv/delta/cap-service.csdl.xml, a copy of the OData metadata document for the CAP service, generated by the cds-to-csdl task.
- The file srv/delta/client-metadata.xml, a client-side copy of the OData metadata document for the delta service. This file may be useful when building client applications as an input to proxy generation tools in the BTP SDKs.
- The file srv/delta/server-metadata.xml, a server-side copy of the OData metadata document for the delta service. This file contains OData annotations indicating the use of delta views and/or cache tables.

The file srv/delta/pom.xml, an Apache Maven POM, generated and then used by the csdl-to-war task to build the delta service WAR file. You can edit this file if you need to customize the Maven build to include additional class libraries, e.g. if you have customized the generated Java code.
The file srv/delta/manifest.yml, a Cloud Foundry App Manifest, generated and then used by the csdl-to-war task to deploy the delta service WAR file. You can edit this file to alter Cloud Foundry runtime options for disk, instances, memory, and so forth. A minimum of 2GB disk and 2GB memory per instance is recommended. Multiple instances can be used if needed to support the client workload.

Customization¶

CDS Annotations¶

Normally mobile back-end tools allows special annotations to be placed in the server-side OData metadata document, to describe options for change tracking, download queries and back-end interactions.

But when using mobile back-end tools to enable change tracking for a CAP service, the OData metadata is not directly maintained. Rather the developer provides a CDS service model (usually together with a CDS data model). Special annotations can be placed in the CDS model to specify options for change tracking and download queries.

Annotations affecting selection criteria for client downloads.
- sync.download - equivalent to the OData SQL.DownloadQuery term.
```
@sync.download : 'select entity.* from Customer entity, CustomerFilter filter where entity.id = filter.customer_id'
entity Customer
{
    key id : Integer;
    name : String(100) not null;
    address : String(100) not null;
    phone : String(30);
}
```
  Note the filter.customer_id in the above example is referring to the foreign key property in the CAP service's OData metadata that is generated by the CAP tools to implement the customer association in the CustomerFilter entity.
- sync.filter - equivalent to the OData SQL.ClientFilter term.
```
@sync.filter
@cds.persistence.skip
entity CustomerFilter
{
    key id : Integer;
    customer : Association to one Customer;
}
```
  Note the additional @cds.persistence.skip annotation to force the CAP tools to not attempt to create the DB artifacts for filter entities; that will be taken care of by the mobile back-end tools.
- sync.inherit - equivalent to the OData SQL.InheritDownloadQuery term.
```
@sync.inherit : order
entity OrderItem
{
    key id : Integer64;
    order : Association to one Order not null;
    product : Association to one Product not null;
    quantity : Integer not null;
    deliveryDate : DateTime;
}
```
Annotations affecting how a cache table is partitioned.
- cache.partition - equivalent to the OData Cache.PartitionBy term.
- cache.byClient - equivalent to the OData Cache.PartitionBy term with a value of client.
- cache.byLocale - equivalent to the OData Cache.PartitionBy term with a value of locale.
Annotations affecting when a cache table is refreshed.
- cache.background - equivalent to the opposite of the OData Cache.OnDemand term, i.e. cached CAP entities will use on-demand refresh by default (with a one hour timeout), and will only use scheduled refresh when explicitly requested. If scheduled refresh is selected, the SAP Business Technology Platform HTTP destination used by the delta service to access the CAP service must specify technical user credentials; this can be configured in the SAP Business Technology Platform Cockpit.
```
@cache.background
entity EntityWithCacheBackground
{
    ...
}
```
- cache.schedule - equivalent to the OData Cache.Schedule term.
```
@cache.schedule : '02:30'
entity EntityWithCacheSchedule
{
    ...
}
```
- cache.timeout - equivalent to the OData Cache.Timeout term.
```
@cache.timeout : 'PT2H'
entity EntityWithCacheTimeout
{
    ...
}
```
- cache.loadAfter - equivalent to the OData Cache.LoadAfter term.
delta.triggers - set to false to explicitly disable the delta views strategy for an entity, thereby forcing use of the cache tables strategy; set to true to explicitly request the delta views strategy for an entity, resulting in a deployment-time error if the CAP entity's definition is such that it cannot be supported by the delta views strategy.
```
@delta.triggers : false
entity MustBeCached
{
    key id : Integer;
    name : String;
}
```
If the delta.triggers annotation is not used, the mobile back-end tools will decide whether to use delta views (if change tracking triggers can be generated) or cache tables (otherwise).

Note that cache tables will be used for CAP entities using localized elements, as well as entities using @restrict with a where condition, since the database views for such entities are too complex for the generation of change tracking triggers. Cache tables will also be used for CAP external entities with @cds.persistence.skip.

Example Model¶

service TestService
{
    @sync.download : 'select entity.* from Customer entity, CustomerFilter filter where entity.id = filter.customer_id'
    entity Customer
    {
        key id : Integer;
        name : String(100) not null;
        address : String(100) not null;
        phone : String(30);
    }

    @sync.filter
    @cds.persistence.skip
    entity CustomerFilter
    {
        key id : Integer;
        customer : Association to one Customer;
    }

    entity Product
    {
        key id : Integer;
        name : String(100) not null;
        description : String(100) not null;
        standardPrice : Decimal not null;
    }

    entity Factory
    {
        key id : Integer;
        name : String(100) not null;
        address : String(100) not null;
        phone : String(30);
    }

    @sync.filter
    @cds.persistence.skip
    entity FactoryFilter
    {
        key id : Integer;
        factory : Association to one Factory;
    }

    @sync.download : 'select entity.* from FactoryProduct entity, FactoryFilter filter where entity.factory_id = filter.factory_id'
    entity FactoryProduct
    {
        key factory : Association to one Factory;
        key product : Association to one Product;
        factoryPrice : Decimal;
    }

    @sync.download : 'select entity.* from Order entity, CustomerFilter filter where entity.customer_id = filter.customer_id'
    entity Order
    {
        key id : Integer64;
        customer : Association to one Customer not null;
        factory : Association to one Factory not null;
        items : Composition of many OrderItem on items.order = $self;
        created : DateTime not null;
        shipped : DateTime;
    }

    @sync.inherit : order
    entity OrderItem
    {
        key id : Integer64;
        order : Association to one Order not null;
        product : Association to one Product not null;
        quantity : Integer not null;
        deliveryDate : DateTime;
    }
}

Last update: July 12, 2022