Modeling Guide for SAP Data Hub

Ingest SQL Server SQL

Microsoft SQL Server Consumer is based on datahub-flowagent and uses Dats Services as an execution engine.

Data Services provides two methods of reading data from Microsoft SQL Server sources. You can either use Microsoft SQL Server Table Consumer to read the table, or use Microsoft SQL Server SQL Consumer to read the result set of the native SQL.

SQL Consumer takes in native SQL statements that you would run on Microsoft SQL Server as input, and it will read the response result set from the source piece-wise (N rows per call) and load to the next operator in line.

Components

  • Triggering the graph:

    constantgenerator1: Provides constant input to consumer, which triggers the ingestion.

  • Defining connection:

    microsoftsqlserversqlconsumer1: Provides consumer connection information.

    MSSQL default port is 1443.

    If no port is provided in connection definition, the default value will be assigned.

  • Loader definition:

    flowagentfileproducer1: Data Services requires a loader, such as CSV Loader, to load to Vflow operators.

Partition

Please see partition configuration on the Oracle SQL Consumer graph (com.sap.dh.ingestion.oracle.sqlconsumer).