Deleting Local Table (File) Records

Delete records from a local table (File) and free up storage through housekeeping on obsolete or already processed data changes.

Prerequisites

To delete local table (file) records, you must have a scoped role that grants you access to a space with one of the following privileges:
  • Data Warehouse Data Integration (--U-E---) – The DW Integrator role template, for example, grants this privilege.

  • Data Warehouse Consumption (--U-E---) – The DW Modeler role template, for example, grants this privilege.

For more information, see Privileges and Permissions and Standard Roles Delivered with SAP Datasphere.

Local Tables (File) store large amount of data, including historical data. When data is updated, a new version is stored in the object store. Keeping all these versions consume a lot of memory, and can affect performance. You must clean them up once they have been consumed by flows to optimize storage costs and improve performance while reading your local tables (file).
  1. From the Data Builder, open your local table (file) in the Table Editor.

  2. Select from the toolbar, (Delete Data From Table).

  3. Choose one of the 2 available options:

    • Delete All Records (Mark as Deleted): Records will not be physically deleted but marked as deleted and filtered out when accessing the active records of the local table. They will still consume storage, and they can still be processed by other apps that consume them.

    • Delete Previous Versions (Vacuum): You delete previous versions, which are older than the number of days you have specified. Records that meet your defined criteria will be permanently deleted. The default value is 7 days. The minimum authorized value is 7 so that records from the last 7 days cannot be deleted, and the maximum value is 999 days. In addition, only records that have been fully processed by all delta consumers (for example transformation flows) can be deleted. Note that, however, even if the data is deleted through a vacuum task, data is kept for another 14 days in the file space storage. As it is very likely that you will not be able to go back more than 30 days, we recommend that you do not enter a period longer than 30 days as a vacuum deletion criterion.For more information, see Create a File Space to Load Data in the Object Store

    [Optional]. When you select one of these options, you can override the default Apache Spark Application Settings that were defined at space creation by your administrator:
    • Use Default: The default application is the application selected in the table settings. If no default application is defined there, the application selected by an administrator during the file space creation is used.

    • Define New Setting for This Table: Select another Apache Spark Application that fits your needs.

  4. Confirm with Delete to start the deletion task.