Acquiring and Preparing Data in the Object Store
Users with a modeler role can load large quantities of data via replication flows and store them inexpensively in file spaces in the SAP Datasphere object store. You can prepare the data using Apache Spark transformation flows and then share data to a standard space to be used as a source of flows, views, and analytic models.
Introduction to the SAP Datasphere Object Store
The object store provides an inbound layer for staging large quantities of data cost-effectively. Data can be loaded by replication flows or pushed through the data product generator for SAP Business Data Cloud. You can optionally further prepare the data with Apache Spark transformation flows. You can then share the tables to standard spaces, where they can be used as sources for flows, views, and analytic models.
Create a File Space in the Object Store
A user with an administrator role can create a space with SAP HANA data lake files storage in the object store. File spaces are intended for loading and preparing large quantities of data in an inexpensive inbound staging area (see Create a File Space to Load Data in the Object Store).
Load Data with Replication Flows
Users with a modeler role can use replication flows to load data in local tables (file) that are stored in a file space (see SAP Datasphere Targets for Replication Flows). A replication flow writes data files to the inbound buffer (specific folder in file storage) of a target local table (File). To process data updates from this inbound buffer to the local table (File), and therefore make data visible, a merge task has to run via a task chain (see Creating a Task Chain) or via the Local Tables (File) monitor. You can monitor the buffer merge status using the Local Tables (File) monitor (See Monitoring Local Tables (File)). You can also automate the merge task at replication flow creation level with the Merge Data Automatically option. For more information, see Creating a Replication Flow.
Integrate SAP BW Data Pushed Through the Data Product Generator
A user with a data integrator role can access local tables (file) received from SAP BW or SAP BW∕4HANA. An SAP BW administrator has pushed BW data into the object store, in a dedicated BW file space, and data is received as local table (file) directly into the Data Builder (see Integrating Data from SAP BW and SAP BW∕4HANA with the Data Product Generator for SAP Business Data Cloud). Updates of these tables are pushed in the inbound buffer. To process data updates from this inbound buffer to the local table (file) generated by SAP BW and therefore make data visible, a merge task has to run via a task chain (see Creating a Task Chain) or via the Local Tables (File) monitor. You can monitor the inbound buffer using the Local Tables (File) monitor (See Monitoring Local Tables (File)).
Prepare Data with Apache Spark Transformation Flows
Users with a modeler role can use Apache Spark transformation flows to clean and otherwise prepare data in a file space (see Creating a Transformation Flow in a File Space). Transformation flows can take one or more local tables (file) as sources, combine them, filter data, add or remove columns, and perform other transformations and output the result to another local table (file) (see Creating a Local Table (File)).