What's New for SAP Cloud Platform

SAP Data Intelligence

2019

Technical Component

Capability

Environment

Title

Description

Type

Available as of

SAP Data Intelligence Data-Driven Insights Cloud Foundry Allow SAP Vora Storage to be 0 It is now possible to deploy an SAP Data Intelligence cluster with no warm storage. Previously, 1 TB of warm storage was required. The 'no warm storage" option is the new default. If you want to activate the warm storage later, you can request a deployment via BCP ticket. New 2019-11-07
SAP Data Intelligence Data-Driven Insights Cloud Foundry Self-Service Data Preparation
  • You can now reorder the actions in your recipe to reflect new needs or maintain existing data refinements. See Manage the Recipe.

  • You can now edit the actions in your recipe to correct or modify an action configuration. See Manage the Recipe.

  • You can now define a formula to change the content or behavior of an existing column. See Change Column Values Using a Custom Action.

  • When you filter your preparation by values in multiple columns, you can now select one of the following options to choose how multiple filter conditions are applied simultaneously:
    • AND

    • OR

    For more information, see Filter Preparation by Column.
New 2019-11-07
SAP Data Intelligence Data-Driven Insights Cloud Foundry ML Scenario Manager Filter on ML Scenarios You can now filter ML scenarios by date or time period. New 2019-11-07
SAP Data Intelligence Data-Driven Insights Cloud Foundry Connection and Metadata Catalog Browser Extension Jupyter Notebook now supports an extension for connection and metadata catalog browsing. New 2019-11-07
SAP Data Intelligence Data-Driven Insights Cloud Foundry HTTP Connection Type Enhancement Connections of type HTTP now have an optional "path" attribute, so that the target URL can be specified with a path prefix. New 2019-11-07
SAP Data Intelligence Data-Driven Insights Cloud Foundry Connection Status Check An issue that caused connection status checks on S3 and other cloud storage to hang for a very long time has been fixed. This issue previously may have caused starvation of resources in the flowaegent application. Fix 2019-11-07
SAP Data Intelligence Data-Driven Insights Cloud Foundry Validating Minio S3 Connection Fails Previously, non-AWS S3 storage would fail if it used a hostname for the connection, and was instead forced to use an IP. This issue has been fixed and either hostname or IP works for S3-compliant storage. Fix 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Training Execution from JupyterLab The SAP Data Intelligence SDK now allows you to create training jobs (including multi-file training jobs) within JupyterLab. New 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Pipeline Engine
  • The SAP Data Intelligence Modeler now shows graphs in stopping status.
  • Resources can now be configured for the Pipeline Engine server and defaults for groups are defined.
  • Dockerfile tags are now passed on to inherited dockerfiles.
  • Storage operators now support SDL connections.
  • There is a new R client operator.
  • Support for Alibaba OSS storage.
  • New SAP HANA operators.
  • For the Streaming Analytics operator, added an “outError” output port to help with error handling and a “committedMessage” output port to help with delivery guarantees in a pipeline.
New 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Node.js Subengine Process Runtime parameter New 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry BW Integration
  • Enable ORC and Parquet as target file formats.
  • Enable delta handling (CDC) in the BW Data Transfer operator.
  • BW Adapter: Enable data staging views for Advanced Data Store Objects.
New 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Cloud Data Integration
  • Filtering for dates.
  • Enable metadata indexing for CDI-based datasets.
New 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Connection Management
  • Private connections via access mode.
  • Auto-create SAP Vora connection with a user and password when a new tenant is created.
  • Connection type for Open API.
New 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Data Workflows
  • Scheduler: Assign schedule ID to runtime instances.
  • Create a standard logging operator.
New 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Policy Management Policy Where Used Check New 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Improvements in Pipeline Scene Editor
  • When you open Pipeline Scene Editor for the first the time, the window alert that informs you that Pipeline Scene Editor is a beta feature is no longer displayed.
  • The Control Editor and the Scene Editor now have the same editor, and when you edit controls, they are added in a playground scene to preview the control.
  • Scripts of controls are no longer copied when edited; instead, they use the base controller.
  • Properties now have validations in the property editor based on their types.
  • Validation errors or errors in scripts are displayed in the new Scene Status Panel in the footer part of Pipeline Scene Editor screen.
Improvement 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Pipeline Engine Fixed Issue The following issues have been fixed in the Pipeline Engine:
  • Read File cannot read all files on bucket root level.
  • Handle arrays with null send to outport.
  • Add quoteIdentifiers option to handle identifiers conflicts.
  • Fix the behavior of the error port handling in SAP Vora ingestor.
  • Update SAP Vora ingestor configuration schemas.
  • Save all contents of dockerfile.
  • Configuration substitution size limit.
  • Include the number of processed records in the status output of SAP Vora ingestor.
  • Force bulk-stmt to bound to a single connection.
  • Streaming Analytics operator “output batch timeout” property issue.
Fix 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Self-Service Data Preparation
  • You can now import data from a comma-separated values (CSV) or Excel (XLSX, XLS) file to create a new preparation. See Upload a File.

  • When you filter your preparation by values in one or more columns, you can now use the following comparison operators:
    • Between

    • Not Between

    • In List

    • Not in List

    For more information, see Filter Preparation by Column.
  • You can now add a column that contains a specific value or values that result from applying an expression to values in other columns in the preparation. See Add Column.

New 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Metadata Explorer New 2019-10-02
SAP Data Intelligence Data-Driven Insights Cloud Foundry Python 2.7 Support

Python 2.7 will no longer be supported in a future SAP Data Hub release. Python 3.6 will continue to be supported.

Python 2.7 will no longer be supported in a future SAP Data Intelligence release. Python 3.6 will continue to be supported.

Announcement 2019-08-29
SAP Data Intelligence Data-Driven Insights Cloud Foundry Metadata Explorer Scorecard Dashboard The rules dashboard organizes your scorecards. Create scorecards to view the quality score of your datasets, categories, and rulebooks. New 2019-08-29
SAP Data Intelligence Data-Driven Insights Cloud Foundry Configure AND/OR for Multiple Rule Conditions In Metadata Explorer, you are now able to define multiple rule conditions that are connected with both AND and OR. New 2019-08-29
SAP Data Intelligence Data-Driven Insights Cloud Foundry Support ORC and Parquet as Preparation Target We have added support for ORC and Parquet as the target dataset format when you execute the recipe of preparation actions on the entire dataset. For more information, see Supported Target Connections and File Types. New 2019-08-29
SAP Data Intelligence Data-Driven Insights Cloud Foundry Resample Dataset After you select data for self-service data preparation from a CSV file, you can re-access a sample of the data and specify header options. For more information, see Specify Column Headers. New 2019-08-29
SAP Data Intelligence Data-Driven Insights Cloud Foundry Reorder Columns During self-service data preparation, you can change the order of columns. The column sequence is then reflected in the output of the result dataset. For more information, see Reorder Columns. New 2019-08-29
SAP Data Intelligence Data-Driven Insights Cloud Foundry Fixed Issue in Self-Service Data Preparation The following issue has been fixed:
  • SAP Note 2808326Information published on SAP site - Readability and usability issues with Default theme in Metadata Explorer.

Changed 2019-08-29
SAP Data Intelligence Data-Driven Insights Cloud Foundry Access a Dataset Stored in External Data Source You can now create an ML pipeline that accesses a dataset stored in an external data source for training purposes. New 2019-08-29
SAP Data Intelligence Data-Driven Insights Cloud Foundry Support Rules for SDL

Use existing categories or create your own categories to hold your rules. Create parameters and define the conditions for the parameters. You can also test the rules to validate whether they pass or fail the rule. Create a rulebook, import the rules, then bind the rules to datasets. Then you can run the rulebook and set thresholds for the data that pass or fail the rule. The rules help you to understand the quality of your data, so that you can improve the data through data preparation or modeling.

New 2019-08-01
SAP Data Intelligence Data-Driven Insights Cloud Foundry File Upload from Metadata Explorer SAP Data Intelligence now enables you to upload data to standard, file-based storage types, including the semantic data lake (see SDL. New 2019-08-01
SAP Data Intelligence Data-Driven Insights Cloud Foundry Bug Fixes

We've made the following improvements:

  • Major stability improvements.

  • You can now use Connection Manager with DataSource tables.

  • If SSL is enabled, you can browse files from HDFS when you create a DataSource table in SAP Vora.

  • Machine learning scenarios can be authored by multiple users. Users can also edit or delete pipelines that were created by another user.

  • We improved the session timeout and activities in JupyterLab.

Changed 2019-08-01
SAP Data Intelligence Data-Driven Insights Cloud Foundry Fixed Issues in Self-Service Data Preparation The following issues have been fixed:
  • SAP Note 2814907Information published on SAP site - The default value for an SAP HANA-wired port in an SAP Vora preparation executions has been corrected.

  • SAP Note 2816637Information published on SAP site - Monitoring page no longer displays the error “Could not read the task execution information from the preparation application” after removal of user access to connection.

Changed 2019-08-01