Change Tracking for CAP Services¶
The tools provided by the SAP Cloud Application Programming Model make it easy to model, generate and deploy an OData service.
If the generated OData service is to be consumed by occasionally-connected mobile clients, together with CloudSyncProvider or OfflineODataProvider in one of the SAP Business Technology Platform client SDKs,
then it is highly beneficial for data synchronization performance to augment the CAP service with OData change tracking capabilities.
In particular, the performance of delta downloads, where the client wishes to download only the back-end data changes since the previous download, can be expected to be achieved in seconds if change tracking is enabled, compared with minutes to hours if change tracking is not enabled (assuming there was a high volume of data in the initial download).
Requirements¶
-
The CAP service is already deployed to the
SAP Business Technology Platform Cloud Foundryenvironment, using either theNode.jsruntime or theJavaruntime. -
The CAP service is already configured to store CAP entities in a SAP HANA Cloud schema (HDI container). CAP entities that proxy/wrap non-CAP back-end APIs and aren't presently stored in the HANA schema can also be supported for change tracking.
Architecture¶
The mobile back-end tools is used to generate and deploy a delta service, which is a proxy service that is positioned between mobile clients and the CAP service. Change tracking is enabled by either delta views or cache tables.
-
For CAP entities (including projections) that are mapped by the CAP tools to simple database views, the mobile back-end tools will generate tracking tables, tracking triggers and delta views (in the same SAP HANA Cloud schema used by the existing CAP service) for change tracking and the delta service will use the delta views to enable efficient OData change tracking.
-
For CAP entities that are mapped by the CAP tools to complex database views (or to external entities, not present in the HANA schema), the mobile back-end tools generates cache tables (in the same SAP HANA Cloud schema used by the existing CAP service) and the delta service uses the cache tables to enable efficient OData change tracking.
Effectively, the mobile back-end tools will use a hybrid of Working with Existing Database Tables and Caching Data in the Cloud (OData Back-End Systems) to enable change tracking for each CAP entity as efficiently as possible.
The generated delta service uses a Java runtime to ensure optimum performance, and should be deployed to run in Apache Tomcat in the SAP Business Technology Platform Cloud Foundry environment.
Development¶
See Installing the Tools for initial setup instructions including the Java Development Kit, Apache Maven and Cloud Foundry CLI. Ensure that the JDK version that you select is at least the minimum version required for CAP Java services.
Create CAP Project¶
An existing CAP project is required, with a service model in the srv subfolder. If you don't have one, you can create a simple CAP project for testing as follows.
It you haven't yet installed the CAP CDS Development Kit, see CAP Getting Started. You can skip the SQLite requirements for CAP. Ensure that node and npm are installed, java and mvn (unless you plan only to use Node.js), and install the CDS Development Kit.
npm i -g @sap/cds-dk
Additionally, to support MTA deployment to SAP Business Technology Platform Cloud Foundry, install the Cloud MTA Build Tool and Cloud Foundry CLI w/ MTA Plugins.
In a Terminal window (or Windows Command Prompt), use the following commands to create, build and deploy your CAP project. The name myapp is a placeholder for your desired CAP project name.
-
Node.js
cds init myapp -
Java
cds init myapp --java
Change the working directory to the newly created project.
cd myapp
Add a CDS data model in the db folder and a CDS service model in the srv folder, for example by adding tiny-sample to your project, and enable the HANA database.
cds add tiny-sample
cds add hana --for production
For simplification of initial testing, you may wish to disable authentication by adding the following to the "cds" section of your project's package.json (for Node.js) or to your project's .cdsrc.json (for Java). It allows your application to be accessed anonymously.
"requires": {
"auth": { "kind": "dummy" }
}
Please remember to enable and test authentication before putting your CAP application into production.
Build and deploy your CAP project:
cds up
Note
If cds up fails because npm ci fails, delete the package-lock.json file and retry.
Create MBT Project¶
Find the cds-delta script in the same folder as the csdl-to-war script.
In a Mac/Linux Terminal or Windows Command Prompt window, change the current directory to your CAP project's root directory, and execute the cds-delta script as follows, where server-odata-sdk-bin is a placeholder for the bin folder in which the cds-delta script is located.
Mac/Linux Terminal:
server-odata-sdk-bin/cds-delta.sh init [ my-service.cds ]
Windows Command Prompt:
server-odata-sdk-bin\cds-delta.bat init [ my-service.cds ]
The square brackets above indicate an optional CAP service CDS file (the brackets should not be included in the command). If the CDS file is not specified, there should be a single CDS file in the CAP project's srv folder.
The cds-delta init command creates a sidecar MBT project within your CAP project's delta subfolder — the generated delta service and the CAP service will run side-by-side in the same SAP Business Technology Platform Cloud Foundry space.
Embedded Mode¶
For the CAP Java runtime (but not the Node.js runtime), the cds-delta init command can be optionally be invoked with the -embed option.
The cds-delta init -embed command creates an embedded MBT project within your CAP project's srv/delta subfolder — the generated delta service and the CAP service will run in the same Java process.
Generation Options¶
The generated delta folder will include a generated options file, e.g. my-service.options.
If you need to add or change any options, please refer to the Service Generator's Command-Line Tool Options.
Convenience Scripts¶
For your convenience after creating the MBT project, two convenience scripts will be generated into your CAP project's root folder, namely cds-delta.bat and cds-delta.sh.
This allows the subsequent build and deploy steps to be conveniently executed without having to remember the path to the server-odata-sdk-bin folder containing the cds-delta script.
Note
If the init step was performed on a Windows system, the cds-delta.sh script might not have the necessary execute permissions for use by Mac/Linux users. The shell command chmod ugo+x cds-delta.sh (on a Mac/Linux system) will apply the necessary permissions, which should be subsequently retained if the project is checked in to a Git repository.
MTA Development Descriptor of CAP Application¶
When you create an MBT project without using the -embed option, the following occurs:
-
A new module named
myapp-deltais automatically added to themta.yamlfile of your CAP application. This addition ensures that the MBT project is built and deployed as a module within the Multitarget Application (MTA). -
The
myapp-deltamodule shares existing resources in themta.yaml, such asmyapp-db,myapp-auth, ormyapp-destination. -
The
srv-apirequire element is added for themyapp-deltamodule. During deployment, thedestinationsenvironment variable, which includes themyapp_srvdestination for service connectivity, is generated.... - name: myapp-delta ... requires: - name: srv-api group: destinations properties: name: myapp_srv url: ~{srv-url}/odata/v4/catalog - name: myapp-db ... -
The generated delta service uses the destination to delegate the
Authorizationheader check to the CAP service or to load cache data from it. Theurlof themyapp_srvdestination is the OData service root URL of the CAP service.- CAP services don't serve documents at the root path ("/") by default. For more information, see CAP: Configures path.
When you use the -embed option during MBT project creation, two things happen:
-
No new module is added to the project. The delta service runs within the
myapp-srvapplication. -
The
srv-apirequire element is automatically added to themyapp-srvmodule.
Build MBT Project¶
After creating the MBT project, and then subsequently each time you change your CAP service model (or any underlying CDS data model), execute the cds-delta script as follows:
Mac/Linux Terminal:
./cds-delta.sh build
Windows Command Prompt:
.\cds-delta.bat build
The cds-delta build command will execute the following commands:
-
cds build
Deploy MBT Project¶
After building the MBT project, and then subsequently each time you change your CAP service model (or any underlying CDS data model), execute the cds-delta script as follows:
Mac/Linux Terminal:
./cds-delta.sh deploy
Windows Command Prompt:
.\cds-delta.bat deploy
The cds-delta deploy command executes the following to deploy a CAP application with the generated delta service:
cds up
You can also directly execute cds up.
Deployment Artifacts¶
To aid understanding of the delta service implementation, please take note of the following generated artifacts.
-
The
db/src/deltafolder, containing the generated HANA deployment (HDI) artifacts for delta views and cache tables. These files supplement the HDI artifacts that were generated by the CAP tools, ensuring that anything needed in the HANA schema to support change tracking will be automatically added during execution of thecds-delta buildcommand. -
The
deltafolder, a sub-project folder containing the main files for the delta service. -
The
delta/srcfolder, containing generated Java source files for the delta service implementation. Customization of the generated Java files is permitted, although it is generally recommended to customize the CAP service implementation files rather than the delta service implementation files (unless the customization is related to the OData change tracking). -
Some OData metadata files. Do not edit any of these files, as your changes will be lost upon any subsequent execution of the
cds-delta buildcommand.-
A copy of the CAP service's metadata file, generated by the
cds-delta buildcommand. -
The file
delta/my-service-client.xml, a client-side copy of the OData metadata document for the delta service. This file may be useful when building client applications as an input to proxy generation tools in the SAP Business Technology Platform Mobile SDKs. -
The file
delta/my-service-server.xml, a server-side copy of the OData metadata document for the delta service. This file contains OData annotations indicating the use of delta views and/or cache tables.
-
- The file
delta/pom.xml, an Apache Maven POM, generated and then used by thecsdl-to-wartask to build the delta service WAR file. You can edit this file if you need to customize the Maven build to include additional class libraries, e.g. if you have customized the generated Java code.
See also: Partitioning for Memory Efficiency.
Delta Service URLs¶
The OData service root URL for your delta service is different for sidecar services and embedded services.
-
If you generated a sidecar service, your delta service will have its own host name (Cloud Foundry route). Use the
cf app myapp-deltacommand to see your app's route(s). Then use the URLhttps://myapp-delta-routeto access your delta service. -
If you generated an embedded service, your delta service will share the host name (Cloud Foundry route) of the CAP service, but with a special URL suffix to identify the delta service. Use the
cf app myapp-srvcommand to see your app's route(s). Then use the URLhttps://myapp-srv-route/delta/MyServiceto access your delta service.
Note
Accessing your delta service requires a Client Registration. See Registration Header.
Delta Service Logs¶
For a sidecar service, the cds-delta init command generates a convenience cf-logs script (delta/logs/cf-logs.sh for Mac/Linux, delta\logs\cf-logs.bat for Windows). Invoking this script from a Visual Studio Code Terminal can be a convenient way to download the logs to a local file, and view them in the editor.
Alternatively for a sidecar service, use standard Cloud Foundry cf logs myapp-delta or cf logs --recent myapp-delta to access your Cloud Foundry logs.
Alternatively for an embedded service, use standard Cloud Foundry cf logs myapp-srv or cf logs --recent myapp-srv to access your Cloud Foundry logs.
See also: Monitoring Generated Services.
Destination Configuration¶
By default, the delta service uses the myapp_srv destination from the destinations environment variable to forward requests to CAP services. Alternatively, you can configure it to use a destination defined in SAP Business Technology Platform through a destination service instance.
Steps to Configure:
- Add a destination to the CAP Application Run the following command to deploy the CAP application to SAP Business Technology Platform Cloud Foundry:
cds add destination
This process adds the myapp-destination service instance.
-
Create a destination in SAP Business Technology Platform Cockpit
- Name: Replace hyphens (-) with underscores (_) in your CAP service's app name. For example, change
myapp-srvtomyapp_srv. - URL: Enter the OData (HTTP) service root URL of your CAP service.
- Authentication: Select
NoAuthentication.
- Name: Replace hyphens (-) with underscores (_) in your CAP service's app name. For example, change
-
Remove
srv-apiRequire Element: Delete thesrv-apirequire element frommta.yaml. This ensures the delta service uses the destination service instead.
Important Note:
The myapp_srv destination in the destinations environment variable takes precedence if it exists. To prioritize the destination service, remove the srv-api require element from mta.yaml.
Customization¶
CDS Annotations¶
Normally mobile back-end tools allows special annotations to be placed in the server-side OData metadata document, to describe options for change tracking, download queries and back-end interactions.
But when using mobile back-end tools to enable change tracking for a CAP service, the OData metadata is not directly maintained. Rather the developer provides a CDS service model. Special CDS annotations can be placed in the CDS model to specify options for change tracking and download queries.
-
Annotations affecting selection criteria for client downloads.
-
sync.download(string) - equivalent to the ODataSQL.DownloadQueryterm.@sync.download : 'select entity.* from Customer entity, CustomerFilter filter where entity.id = filter.customer_id' entity Customer { key id : Integer; name : String(100) not null; address : String(100) not null; phone : String(30); }Note the
filter.customer_idin the above example is referring to the foreign key property in the CAP service's OData metadata that is generated by the CAP tools to implement thecustomerassociation in theCustomerFilterentity. -
sync.filter(boolean) - equivalent to the ODataSQL.ClientFilterterm.@sync.filter @cds.persistence.skip entity CustomerFilter { key id : Integer; customer : Association to one Customer; }Note the additional
@cds.persistence.skipannotation to force the CAP tools to not attempt to create the DB artifacts for filter entities; that will be taken care of by the mobile back-end tools. -
sync.inherit(path) - equivalent to the ODataSQL.InheritDownloadQueryterm.@sync.inherit : order entity OrderItem { key id : Integer64; order : Association to one Order not null; product : Association to one Product not null; quantity : Integer not null; deliveryDate : DateTime; }
-
-
Annotations affecting whether a cache table is used.
-
cache.required(boolean) - specifies that an entity should be cached even if delta views would otherwise be selected by default.Note that cache tables will be used for CAP entities using
localizedelements, as well as entities using@restrictwith awherecondition, since the database views for such entities are too complex for the generation of change tracking triggers. Cache tables will also be used for CAP external entities with@cds.persistence.skip.
-
-
Annotations affecting when a cache table is refreshed.
-
cache.background(boolean) - equivalent to the opposite of the ODataCache.OnDemandterm, i.e. cached CAP entities will use on-demand refresh by default (with a one hour timeout), and will only use scheduled refresh when explicitly requested. If scheduled refresh is selected, the SAP Business Technology Platform HTTP destination used by the delta service to access the CAP service must specify technical user credentials; this can be configured in the SAP Business Technology Platform Cockpit.@cache.background entity EntityWithCacheBackground { ... } -
cache.schedule(time) - equivalent to the ODataCache.Scheduleterm.@cache.schedule : '02:30' entity EntityWithCacheSchedule { ... } -
cache.timeout(duration) - equivalent to the ODataCache.Timeoutterm.@cache.timeout : 'PT2H' entity EntityWithCacheTimeout { ... } -
cache.loadAfter(string) - equivalent to the ODataCache.LoadAfterterm. -
cache.loadBefore(string) - equivalent to the ODataCache.LoadBeforeterm.
-
-
Annotations affecting how a cache table is partitioned.
-
cache.partition- equivalent to the ODataCache.PartitionByterm. -
cache.byClient(boolean) - equivalent to the ODataCache.ByClientterm. -
cache.byLocale(boolean) - equivalent to the ODataCache.ByLocaleterm.
-
Example CDS Model¶
service TestService
{
@sync.download : 'select entity.* from Customer entity, CustomerFilter filter where entity.id = filter.customer_id'
entity Customer
{
key id : Integer;
name : String(100) not null;
address : String(100) not null;
phone : String(30);
}
@sync.filter
@cds.persistence.skip
entity CustomerFilter
{
key id : Integer;
customer : Association to one Customer;
}
entity Product
{
key id : Integer;
name : String(100) not null;
description : String(100) not null;
standardPrice : Decimal not null;
}
entity Factory
{
key id : Integer;
name : String(100) not null;
address : String(100) not null;
phone : String(30);
}
@sync.filter
@cds.persistence.skip
entity FactoryFilter
{
key id : Integer;
factory : Association to one Factory;
}
@sync.download : 'select entity.* from FactoryProduct entity, FactoryFilter filter where entity.factory_id = filter.factory_id'
entity FactoryProduct
{
key factory : Association to one Factory;
key product : Association to one Product;
factoryPrice : Decimal;
}
@sync.download : 'select entity.* from Order entity, CustomerFilter filter where entity.customer_id = filter.customer_id'
entity Order
{
key id : Integer64;
customer : Association to one Customer not null;
factory : Association to one Factory not null;
items : Composition of many OrderItem on items.order = $self;
created : DateTime not null;
shipped : DateTime;
}
@sync.inherit : order
entity OrderItem
{
key id : Integer64;
order : Association to one Order not null;
product : Association to one Product not null;
quantity : Integer not null;
deliveryDate : DateTime;
}
}
Security¶
Default Behavior (No Authentication)¶
If no authentication module is added:
- The
-login noneoption is automatically included in the Generation Options File. - The delta service allows anonymous access.
XSUAA Authentication¶
When XSUAA is added (cds add xsuaa):
- The
-login xsuaaoption appears in the Generation Options File. - A valid XSUAA access token is required in the HTTP
Authorizationheader. - Unauthenticated requests receive a 401 response.
Delegate Authentication to CAP Service¶
To delegate authentication to a CAP service, follow these steps:
- Modify the login option in the Generation Options File to
-login http@myapp_srv. - The delta service automatically validates the Authorization header by delegating to the CAP service through the
myapp_srvdestination. - No additional security configuration is required for the delta service.
Delegation to the CAP service is also used for permission checks when the client requests are OData queries. To check if the user has read access to an entity set, the delegated permission check will use a query such as SomeEntitySet?$top=1.
If the CAP service performs user-specific filtering of data even when the client does not specify an OData $filter in the query URL, then you should use CDS annotations @cache.required and @cache.byClient (noting that this will result in the cache containing user-specific copies of data), or for entities with shared data using delta views, the CDS annotation @sync.download could be used to replicate the user-specific filtering of that shared data.