Modeling Guide for SAP Data Hub

GCS

The Data Generator produces arbitrary data.

The data is written to files on GCS by Write File. The files are stored in the directory ./tmp/demo/ (where ./ is the current working directory, i.e. where the Data Pipeline Engine was started). The file names follow the scheme file_<counter>.txt.

The Read File reads this file and the content is written to a terminal.

Prerequisites

  • You need your GCS credentials (the access key), the project ID, and the bucket.

Configure and Run the Graph

Follow the steps below to run the example from the Data Pipeline UI:
  1. In the left panel, select the Graphs tab and navigate to com/sap/demo/s3.
  2. Check the connection configuration of the Write File node: gcsJsonKey, gcsProjectID, gcsBucket.
  3. Check the configuration of the Read File node: gcsJsonKey, gcsProjectID, gcsBucket.
  4. In the tool bar, select Run (play button).
  5. The Status panel indicates if the graph is running.
  6. Use the context menu Open UI of the Terminal node to open the terminal.
  7. The terminal opens and you see the produced files and their content.