Automating Data Movement, Transformation, and Persistence Tasks

Users with a modeler role can create task chains to perform sequences of tasks, while users with an integrator or space administrator role can create schedules to regularly run and monitor individual tasks and task chains.

This topic contains the following sections:

Run Sequences of Tasks with Task Chains

Users with a modeler role can create task chains to run sequences of tasks. See Creating a Task Chain.

By default, in a simple linear chain, one task must complete before the next task can begin. This is ideal when running a sequence of dependent tasks, such as:

Run a replication flow that replicates data to a local table.
Run a transformation flow that takes the table's data, transforms it, and then writes the results to a second local table.
Persist data in a view for which the second table is a source.

Task chains also provide various ways of flexibly grouping and automating tasks:

If appropriate, you can run tasks in parallel branches in your task chain and specify whether the run should continue once all or any of the branches has completed. See Run Parallel Tasks in a Task Chain.
You can embed one or more task chains (including task chains shared from another space) in a task chain, which will then run them as tasks. See Nest and Share Task Chains.
You can pass input parameters to your task chain and provide these input parameter values to objects as their tasks are run. Create Input Parameters in Task Chains.
You can automate the sending of emails when the task chain or individual tasks complete. See Configure Email Notification for Task Chains and Notification Tasks.

In addition to running flows and managing other SAP Datasphere objects, you can also add tasks to your task chains to:

Run procedures in Open SQL schemas. See Run Open SQL Procedures in a Task Chain.
Make calls to external REST APIs. See Run API Tasks in a Task Chain.
Run process chains in your SAP BW bridge tenant. See Run SAP BW Bridge Process Chains in a Task Chain.

Schedule, Monitor, and Debug Data Movement Tasks

Users with an integrator or space administrator role can create schedules to run tasks and can monitor and debug these runs.

For example, you can choose to run your task or task chain hourly, daily, weekly, or monthly (including via a cron expression), depending on how frequently your data changes and how fresh you need it to be. See Scheduling Data Integration Tasks and Managing and Monitoring Data Integration.

You can use the Data Pipeline Analyzer to identify, for any analytic model, view, or local table, how data flows to the object and any issues in the pipeline. See Data Pipeline Analyzer.