Modeling Guide

S3

The data generator produces arbitrary data.

The data is written to files on S3 by Write File. The files are stored in the directory /tmp/demo/ (of the specified bucket). The file names follow the scheme file_<counter>.txt.

The Read File reads these files and the content is written to a terminal.

Prerequisites

  • You need your AWS credentials, the bucket, and the region to use.

Configure and Run the Graph

Follow the steps below to run the example from the Data Pipeline UI:
  1. In the left panel, select the Graphs tab and navigate to com/sap/demo/s3.
  2. Check the connection configuration of the Write File node: awsAccessKeyId, awsSecretAccessKey, bucket, region.
  3. Check the configuration of the Read File node: awsAccessKeyId, awsSecretAccessKey, bucket, region.
  4. In the tool bar, select Run (play button).
  5. The Status panel indicates if the graph is running.
  6. Use the context menu Open UI of the Terminal node to open the terminal.
  7. The terminal opens and you see the produced files and their content.

Optional Step: Check the files on S3

  1. Connect to S3 and browse the directory /tmp/demo/ (of the specified bucket) to see the produced files.