Modeling Guide

Create a Data Pipeline Task

Creating and executing a data pipeline task helps execute a data pipeline in an SAP Vora system.

Prerequisites

  • You have created a project.
  • You have created and activated at least one destination in the same project in which you are creating the data pipeline task.
  • The destination has a connection (connection type: SAP Data Hub Pipeline) to an SAP Vora system (system type: SAP VORA).

Context

A data pipeline represents a concrete and complex data flow, and helps transform data between elements connected in a series. The data pipeline task in SAP Data Hub, when executed, can help process the raw data from multiple sources and make it available for different use cases.

Procedure

  1. Start the SAP Data Hub cockpit in a Web browser.
  2. In the System Status section, choose the Modeling tile.
    The cockpit opens the SAP Data Hub Modeling tool in a new tab in the same browser window.
  3. Create task.
    1. In the navigation pane, right-click the project within which you want to create the task and choose Start of the navigation pathNew Next navigation step TaskEnd of the navigation path.
    2. In the Create Task dialog box, provide a name for the task.
    3. In the Type dropdown list, select Data Pipeline.
    4. Choose Create.
      The tool opens a new editor where you can define your task.
  4. Define the task.
    1. In the Destination value help list, select the required destination.
      The tool populates the value help list with destinations that have connections to an SAP Vora system. The connection type is, SAP Data Hub Pipeline.
    2. In the Description text field, provide a description for the task.
    3. In the Data Pipeline value help list, select the required data pipeline.
      The tool populates the value help list with graphs from the selected destination. Once you select a graph, it displays the configuration parameters defined for the graph.
    4. Use the Running Permanently toggle button to indicate whether the data pipeline referenced in the task is a permanently running data pipeline.

      Toggle Status

      Description

      YES

      If set to YES, the task execution checks whether the data pipeline state is in running, starts the data pipeline execution if it is not in running, and terminates immediately (status:completed) while the data pipeline remains in the running state. But, if the task was already running with a different task version, then that instance is stopped, and the new task version is started and immediately terminated (status: completed).

      NO

      If set to NO, the task executes the data pipeline once, and the task execution terminates after the pipeline execution terminates. 

  5. (Optional) Edit configuration parameters.
    1. If you want to edit the configuration parameters defined for the graph, in the Parameter Values section, choose to edit the parameter values.
  6. (Optional) Preview selected graph.
    If you want to preview the graph that you have selected,
    1. In the editor toolbar, choose the Graphs tab.
      The tool opens the selected graph in a read-only mode.
    2. If you want to open the graph in the data pipeline editor, choose (Open in Data Pipeline Editor).
  7. Save changes.
    In the global toolbar, choose Save to save the task.
  8. Activate the task.
    After creating a task object, activate the task. Activation is necessary to convert the design-time object to its equivalent runtime object in the database.
    1. In the global toolbar, choose (Activate) to activate the task.
  9. (Optional) Execute the task.
    After creating and activating a task, you can directly execute the task for an immediate execution.
    1. In the global toolbar, choose (Execute) to execute the task.
      The tool opens a new editor, where you can monitor the status of the task execution. You can also monitor the status of these task executions in the SAP Data Hub cockpit (monitoring dashboard).
  10. (Optional) Abort task.
    If you want to abort the execution of any task in the task workflow,
    1. On the task node, choose (Trigger one of several options available).
    2. Select the Cancel Task menu option.
    3. In the editor toolbar, choose (Refresh) to refresh and view change in the task status.