Modeling Guide for SAP Data Hub

Ingest OData Query

OData Query Consumer is based on datahub-flowagent and uses Dataservices as execution engine.

Dataservices provides reading data from OData sources using OData Query Consumer to read the result set of the native OData Query URI.

Query Consumer takes in an OData Query URI that you would run on OData as input and it will read the response result set from the source piece-wise (100 rows per call) and load to the next operator in line.

Example:

  • Entity set: Orders

  • Projections: Orders?$select=CustomerID,EmployeeID,Employee/LastName,Employee/FirstName&$expand=Employee

  • Filters: Orders?$filter=Employee/FirstName eq 'Steven'

Components

  • Defining OData connection

    ODataqueryconsumer1: provides OData consumer connection information

  • Flowagent Producer

    flowagentfileproducer1: Flowagent based consumer requires a flowagent based producer like file, database or csv producer. The output of the odata is written to /vrep/odata_<sourceTableName>.csv

  • Terminating Graph

    graphterminator1: Once the flowagent producer writes the output to the file, it will terminate the graph.