Modeling Guide for SAP Data Hub

Operator Details

Every operator has an ID (also known as name) and a title (also known as the description). The ID is its unique identifier, with a strict format. The title is what the graphical interface displays.

All the operators available when creating a graph are known as extensions (referred to as simply operators). They extend base operators, which are visible when creating a new operator.

The extension is expressed by a file in the Modeler file system. This file must be named operator.json and its folder hierarchy must match its ID. This is automatically done when an operator is created following Create Operators.

E.g.:
ID: 'com.sap.foo.bar'
Filepath: './operators/com/sap/foo/bar/operator.json'

Operator JSON

The operator.json file is responsible for the definition of the operator, including the graphical interface information. It has the following structure (mandatory fields are denoted by *):
  • description: the operator title;

  • *icon: the operator icon as a Font Awesome icon name available at https://fontawesome.com/icons/Information published on non-SAP site;

  • *iconsrc: the path to the SVG icon file - path is relative to the operator.json;

  • *component: the base operator ID to be extended;

  • inports: an array of input ports;

  • outports: an array of output ports;

  • config: a map of configuration parameters, mapping a configuration parameter ID to its default value;

  • *config.$type: $type field pointing to its schema. More...

  • tags: a map of tags, mapping each tag ID to its default value;

  • enableportextension: a boolean value that, if set to true, allows adding additional ports and configurations to the operator through the UI;

  • extensible: a boolean value that, if set to true, allows a base operator to be extended;

The operator.json should result in the following structure:

{
    "description": "<operator-title>",
    "icon": "<fontawesome-icon>",
    "iconsrc":"<icon-file>",
    "component": "base-operator-id",
    "inports": [
        {
            "name": "<inport1-id>",
            "type": "<inport1-type>"
        },
        ...
    ],
    "outports": [
        {
            "name": "<outport1-id>",
            "type": "<outport1-type>"
        },
        ...
    ],
    "config": {
        "<config-id>": "<config-value>",
        ...
    },
    "tags": {
        "<tag-id>": "<tag-value>",
        ...
    },
    "enableportextension": <true/false>,
    "extensible": <true/false>,
}

Example:

{
    "iconsrc": "read.svg",
    "component": "com.sap.storage.read",
    "config": {
        "$type": "http://sap.com/vflow/com.sap.storage.read.schema.json#"
    },
    "tags": {},
}

Naming

The documentation is presented in a README file in Markdown format.

The file must be named README.md and placed at the same operator.json folder if the documentation only makes sense for the extension - for multiple subengine implementations, where we have multiple operator.json, each of them should have a README in the same folder.

Documentation

The documentation is presented in a README file in Markdown format.

The file must be named README.md and placed at the same operator.json folder if the documentation only makes sense for the extension - for multiple subengine implementations, where we have multiple operator.json, each of them should have a README in the same folder.

README structure

It should follow the structure:
<operator-title>
===

<introduction>

<links-to-examples>

Configuration parameters
---

- <configuration-parameter-1>
- <configuration-parameter-2>
- ...

Input
---

- <input-port-1>
- <input-port-2>
- ...

Output
---

- <output-port-1>
- <output-port-2>
- ...
If an item list (parameters or ports) is empty, a single item None must be listed. E.g.:
Configuration parameters
---

- None

Introduction

Text with the content as follows:
- **configuration-id** 
    (type <configuration-type>, default: <configuration-default>) 
    <!-- mandatory: only if applicable --> mandatory:
    <!-- brief description --> Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
    <!-- if needed, link to document with further description -->
    (Details are described here)[<link-to-config-docs>].

    - ID: `<configuration-id>`
    - Type: `<configuration-type>`
    <!-- Default: a value must be expressed according to its type formatting,
        e.g.:
            string -> `"value"`,
            int -> `42`,
            object -> `{ "k": ["v1, "v2"] }`,
            ...
    -->
    - Default: `<default-configuration-value>`
    <!-- Possible values: only valid for "enum" type -->
    - Possible values:
        - `<value-1>`
        - `<value-2>`
    <!-- Expected input: only valid when the "pattern" is set -->
    - Expected input: `<pattern-regex>`
    <!-- Additional specification fields may be provided -->
This should result in something like the following:
  • Connection Protocol [mandatory]

    Protocol used in the request to the service.

    • ID: connProtocol

    • Type: string

    • Default: "HTTP"

    • Possible values:

      • "HTTP"

      • "HTTPS"

Ports

Ports are identified by a unique ID (aka name).

They're formatted in the following way:
- **<port-id>** (type <port-type>): Express the parameter.
    If further is needed, document it in a separate file, and [link]()
    to it in this sentence. External documentation may be linked.

Configuration

Parameter naming follows the same standard as operator naming.

Configuration schema

A schema for the configuration must be given. It contains each parameter and further constraints for the UI.

It must be linked in the operator.json, like:
{
    ...
    "config": {
        "$type": "<$id-from-schema>"
    },
}
Each parameter in the schema must:
  • point to its ID with the object's key;

  • have a set title;

  • use the most strict type;

  • have a validation regex in pattern, if applicable;

  • be listed as required if applicable.

The schema may be written manually or with the help of the Types panel of the graphical interface. It should either be saved as:
  1. configSchema.json in the same operator.json folder;

  2. schema.json in /types/<operator-id>/schema.json.

Approach 1. should be considered first (and is the one taken by the graphical interface tool).

A brief example of the schema is:
{
    "$schema": "http://json-schema.org/draft-06/schema#",
    "$id": "http://sap.com/vflow/<operator.id>.schema.json",
    "title": "<schema-title>",
    "description": "<schema-description>",
    "type": "object",
    "properties": {
        "user": {
        "title": "User",
        "type": "string",
        "secure": true
        },
        "password": {
        "title": "Password",
        "type": "string",
        "secure": true,
        "format": "password"
        },
        "fooConnectionID": {
            "title": "Foo Connection ID",
            "description": "Connection ID used to connected to Foo",
            "type": "string",
            "format": "com.sap.dh.connection.id"
        },
        "isFoo": {
            "title": "Is Foo",
            "type": "boolean",
            "description": "Determines if operator is Foo.",
        },
        "reqMode": {
            "title": "Request Mode",
            "type": "string",
            "enum": [
                "Foo",
                "Bar"
            ]
        },
        "noOfReqs": {
            "title": "Number of Requests",
            "type": "integer",
            "description": "Number of Bar requests to be done.",
            "sap_vflow_constraints": {
                "ui_visibility": [
                    {
                        "name": "reqMode",
                        "value": "Bar"
                    }
                ]
            }
        },
        "filepath": {
            "title": "File path",
            "type": "string",
            "description": "File path to save request. Must start with '/'.",
            "pattern": "^\/.*$"
        }
    }
}

Schema Types

The available types for a configuration parameter are:
"array", "boolean", "integer", "number", "object", "string"

Find out more at http://json-schema.org/draft-06/schemaInformation published on non-SAP site

Category

An operator is recommended to belong to at least one category, and may belong to more than one. A new operator can be added to the relevant categories, and new categories can also be created. This can be done manually, or through the graphical interface. To do it manually (this is what the graphical interface does):
  • on the Repository tab, open ./general/ui/*.settings.json;

  • in the object "operatorCategories" ("graphCategories" for a graph), add your operator ID to the array of an existing category or create a new one. The category object has the following structure:
    {
        "name": "Category Name",
        "entities": [
            "com.sap.foo.bar",
            "com.sap.operator.test"
        ]
    }

The category name follows the same operator title naming constraints, already mentioned here.