Modeling Guide for SAP Data Hub

Vora Loader

The Vora Loader operator works as a client for the SAP Vora transaction coordinator and allows you to load data into SAP Vora.

For now, loading data is supported through HDFS, WEBHDFS, S3, ADL, and GCS.

Configuration Parameters

Parameter

Type

Description

voraConnection

object

The object containing the connection parameters to a Vora instance.

voraConnectionTimeoutInMs

int

The number of milliseconds to wait when connecting to Vora before timing out.

Default: 60000

initStatements

string

SQL code to be executed when the operator is initialized.

Default: ""

tableName string Mandatory. The name of the table where the loaded file is inserted.

Default: "TEST.T1"

fileFormat

string

The format of the file to be loaded.

Default: "csv"

columnOptions array A JSON array containing attributes that can be added to a specific column for CSV files. It is composed by the following values:
columnName (type string)
Mandatory. Name of the column to add attributes.
columnIndex (type int)
Index that will represent the column.
decimalSeparator (type string)
Character that corresponds to the decimal separator.
thousandsSeparator (type string)
Character that corresponds to thousands separator.
format (type string)
Format of dates or datetimes.
nullValue (type string)
Value that corresponds to null.
escapeCharacter (type string)
Character that should be escaped.

delimiter

string

Mandatory. The character used to separate column values in CSV files.

Default: ","

skip int The number of lines at the beginning of the CSV file to skip.
stringDelimiter string The delimiter for strings in CSV files.
decimalSeparator string Character that corresponds to the decimal separator in CSV files.
thousandsSeparator string Character that corresponds to thousands separator in CSV files.
dateFormat string Format of dates of the datasource, by default vora uses 'YYYY-DD-MM' in CSV files.
format string Format of dates or datetimes of the datasource in CSV files.
nullValue string Value that corresponds to null on the datasource in CSV files.
escapeCharacter string Character that should be escaped in CSV files.

storage

string

The service to load files from. Additional parameters may depend on the selected service. Local file system refers to the file system where the Vora instance is running.

Default: "local"

proxy

string

The proxy from storage. Currently only supported on WebHDFS, S3, WASB and GCS connections.

Default: "noproxy"

numRetryAttempts

int

The number of retry attempts.

Default: 0

retryPeriodInMs

int

The waiting time in milliseconds between consecutive retry attempts.

Default: 0

Input

Input

Type

Description

inFilename

string

The storage path of the file to be loaded to the transaction coordinator.

Output

Output

Type

Description

outResult

string

Either "Successfully executed sql!" or the appropriate error message.