Modeling Guide

Create a File Operation Task

Creating and executing a file operation task helps perform file operations such as copy and delete on data sets.

Prerequisites

  • You have created a project.
  • You have created and activated at least one destination in the same project in which you are creating the file operation task.
  • The destination has a connection (connection type: Hadoop Distributed File System (HDFS), Amazon S3) to an SAP Vora system (system type: SAP VORA).
  • You have created and activated one or more data sets with these destinations.

Context

The data sets on which you perform the file operation tasks are abstractions of actual data or structures that reside in an SAP Vora. You can create data sets for files in HDFS or Amazon S3, or for tables in SAP Vora.

Procedure

  1. Start the SAP Data Hub cockpit in a Web browser.
  2. In the System Status section, choose the Modeling tile.
    The cockpit opens the SAP Data Hub Modeling tool in a new tab in the same browser window.
  3. Create task.
    1. In the navigation pane, right-click the project within which you want to create the task and choose Start of the navigation pathNew Next navigation step TaskEnd of the navigation path.
    2. In the Create Task dialog box, provide a name for the task.
    3. In the Type dropdown list, select File Operation Task.
    4. Choose Create.
      The tool opens a new editor where you can define your task.
  4. Define the task.
    1. In the File Operation dropdown list, select the required file operation type.
    2. In the Description text field, provide a description for the task.
    3. In the Source value help list, select the required source data set.
      If you have selected the Delete file operation, provide only the target data set.
    4. In the Target value help list, select the required target data set.
      The tool populates the Source and Target value help lists with data sets from the same project. These data sets are defined with destinations that have connections to an SAP Vora system (connection type: HDFS, or Amazon S3).
    5. For the Copy Data file operation, in the Mode dropdown list, select the required value.

      Mode

      Description

      Append

      Copy data by adding the source data set with the target data set.

      Overwrite

      Copy data by replacing the source data set with the target data set.

  5. Save changes.
    In the global toolbar, choose Save to save the task.
  6. Activate the task.
    After creating a task object, activate the task. Activation is necessary to convert the design-time object to its equivalent runtime object in the database.
    1. In the global toolbar, choose (Activate) to activate the task.
  7. (Optional) Execute the task.
    After creating and activating a task, you can directly execute the task for an immediate execution.
    1. In the global toolbar, choose (Execute) to execute the task.
      The tool opens a new editor, where you can monitor the status of the task execution. You can also monitor the status of these task executions in the SAP Data Hub cockpit (monitoring dashboard).
  8. (Optional) Abort task.
    If you want to abort the execution of any task in the task workflow,
    1. On the task node, choose .
    2. Select the Cancel Task menu option.
    3. In the editor toolbar, choose (Finished) to refresh and view change in the task status.