Understanding Different Methods for Extracting Metadata

Depending on the type of source system connected to the catalog, metadata for objects is extracted automatically, as a scheduled task, by a background process, or manually. These different methods help you ensure that the content in the catalog is up-to-date.

Automatic Extraction

The catalog monitors changes to objects in the connected SAP Datasphere and SAP Analytics Cloud source systems. When changes are detected, the catalog extracts metadata for those objects and updates the corresponding assets in real time. The catalog can also detect when objects are added or removed.

The following types of changes are detected and the catalog is updated accordingly:

Source System Change	Automatic Catalog Action
A new object is created.	A new catalog asset is created and the functional status is set to Current. If the asset is automatically published, it'll be discoverable in the catalog. Users with the Catalog Administrator role can enrich the metadata for the asset as needed. If the asset is not automatically published, users with the Catalog Administrator role can enrich the metadata and prepare the asset before publishing.
An existing object is updated with changes.	The existing catalog asset's metadata is updated with changes and the functional status label Current is applied.
An existing object is deleted.	The catalog asset status label Unavailable is applied. Users with the Catalog Administrator role can clean up and remove the asset from the catalog. If the deleted object is referenced and used by another asset in the catalog, the status label Remotely Deleted is applied.

Source System Change

Automatic Catalog Action

A new object is created.

A new catalog asset is created and the functional status is set to Current.

If the asset is automatically published, it'll be discoverable in the catalog. Users with the Catalog Administrator role can enrich the metadata for the asset as needed.
If the asset is not automatically published, users with the Catalog Administrator role can enrich the metadata and prepare the asset before publishing.

An existing object is updated with changes.

The existing catalog asset's metadata is updated with changes and the functional status label Current is applied.

An existing object is deleted.

The catalog asset status label Unavailable is applied. Users with the Catalog Administrator role can clean up and remove the asset from the catalog.

If the deleted object is referenced and used by another asset in the catalog, the status label Remotely Deleted is applied.

Synchronization Schedule

The catalog does not monitor changes to objects in BW systems (SAP Datasphere, SAP BW bridge, SAP BW∕4HANA, or SAP BW 7.5). To ensure that objects from these source systems are current, you can create a synchronization schedule to extract the metadata (see Manually Connecting a BW System). Because the sychronization schedule does not run by detecting changes in the objects, the updates are not in real time.

Background Process

The catalog runs a background process on a regular basis to extract metadata for data products in SAP and partner systems that are part of SAP Business Data Cloud formations (for example, SAP S/4HANA or SAP Databricks). For example:

For SAP systems, new data products are added to the catalog shortly after they are activated.
For partner systems, a process is run every 5 minutes. This process polls the system and adds new data products that are created.
For all systems, a background process is run every 24 hours to update the data products in the catalog. Data products that have been deleted or deactivated are removed from the catalog.

Manual Synchronization

When one of the metadata extraction methods fails to complete or the metadata extraction for several objects encounters errors, you might need to manually synchronize the source system. Manually synchronizing a source system can take considerable time and resources, so you should run a manual synchronization only in the certain situations (see Manually Synchronizing a System).

Next Steps

Now you know the different ways source systems are synchronized with the catalog. After a synchronization is completed, you can check the catalog to see if the data or assets you were expecting are there. If anything is missing, look for the following issues:

In the source system, check that the authorized user has the necessary permissions to access the source objects.
On the Metadata Extractions page, check that the connected source system has an authorized user.
For source objects in SAP Datasphere, check that they have been saved and deployed (see Saving and Deploying Objects).
On the source system details page, review the Extraction Logs tab and search for the source objects to find their log entries (see Monitoring Metadata Extractions).