Modeling Guide for SAP Data Hub

Maintain Resource Requirements for Graphs

You can specify compute resource requirements (memory or CPU) for graph groups.

For each resource, you can specify the request and limit properties. The request property specifies the initial resource quantity that the application requires to start the group execution. If there is not enough resource available, then graph execution fails to start. The limit property specifies limit for a resource usage.

If memory limit is violated, then group execution is terminated. The cpu limit, in turn, helps to limit CPU consumption in case of excessive CPU usage by a group execution. Resource requirements use the same notation as Kubernetes to specify resource quantity. For more information on memory limit and cpu limit, see the Kubernetes documentation:

memoryInformation published on non-SAP site

cpuInformation published on non-SAP site

Specifying Resource Requirements for a Graph

Resource requirements are specified in graph description (graph.json). The groupResources property provides default resource requirements for all groups. For a group (except default), you can overwrite requirements in resources property in the group definition.

Here is an example of modified data generator demo graph (unnecessary details are omitted):

In this example graph, there are two groups: default (contains terminal operator) and group1 (contains dataGenerator operator). Resource requirements from groupResources are applied to default group:

First, the resource requirements from groupResources property is applied to group1. Then, all resource requirements are overwritten for group1 but not memory request (since it's not specified for group1):