MySQL Sources for Replication Flows
You can use MySQL connections as sources in replication flows to replicate data into supported targets. This feature uses the Connectivity Framework and supports the Initial Only load type.
Prerequisites
-
Make sure you have created a MySQL connection in Connection Management and that it is available in your space. For more information, see MySQL Connections
-
Only tables that contain a primary key are supported as replication sources.
-
Only the Initial Only load type is supported. Initial and Delta and Delta Only load types are not supported.
Restrictions
-
Source settings are not available for this source type.
-
Custom parameters are not supported.
-
Connections through load balancers are not supported.
Partitioning Timeouts for Large MySQL Tables
If a partitioning timeout occurs, try the following in order:
-
Stale table statistics: MySQL statistics in information_schema can take time to update. Stale statistics lead to incorrect group calculations and a partitioning timeout. Run the following statement, then rerun the replication ANALYZE TABLE <tableIdentifier>;flow:
-
Persistent timeout – InnoDB tuning: If the timeout persists, review the following MySQL parameters to ensure the index fits in memory and I/O is tuned for sequential large-table reads:
Parameter
Recommended Value
innodb_buffer_pool_size
70–80% of RAM (dedicated server); at minimum, twice the index size
innodb_buffer_pool_instances
1 per GB of buffer pool; maximum 8
innodb_read_io_threads
8 for SSD; 16 or more for NVMe (restart required)
innodb_read_ahead_threshold
0 for sequential replication scans
read_rnd_buffer_size
32 MB for large table reads
max_connections
Set conservatively; each idle connection uses approximately 1 MB
-
Continued DEADLINE_EXCEEDED errors – covering index: On very large tables, create the SAP_MYSQL_REPLICATION_PK index to allow the connector to scan only the primary key columns instead of the full clustered index. The index must cover exactly the primary key columns and nothing else:-- Replace pk1, pk2, ... with the actual primary key column names in declaration order. CREATE INDEX SAP_MYSQL_REPLICATION_PK ON `schema`.`table` (`pk1`, `pk2`, ...);
-
Index creation runs online but takes time proportional to table size. Monitor progress with:SELECT STAGE, WORK_COMPLETED, WORK_ESTIMATED FROM performance_schema.events_stages_current;
-
After creation, verify the index is used:EXPLAIN SELECT `pk1`, ..., `pkN` FROM `schema`.`table` ORDER BY `pk1`, ..., `pkN` LIMIT 1 OFFSET 1000000; -- Expected: key = SAP_MYSQL_REPLICATION_PK, Extra = Using index
-
Verify the index size fits within your innodb_buffer_pool_size:SELECT INDEX_NAME, ROUND(stat_value * @@innodb_page_size / 1024 / 1024 / 1024, 2) AS size_gb FROM mysql.innodb_index_stats WHERE database_name = '<schema>' AND table_name = '<table>' AND stat_name = 'size';
-
Replicating Source Objects Without Primary Keys
Replicating source objects without a primary key is not supported. Only source objects that have a primary key defined can be used as source objects in a replication flow.
Data Types
|
MySQL Data Type |
SAP Datasphere |
|---|---|
|
BIGINT |
int64 |
|
BIGINT UNSIGNED |
uint64 |
|
BINARY(n), VARBINARY(n), 1≤n≤5000 |
binary(n) |
|
BINARY(n), VARBINARY(n), n > 5000 |
Not Supported |
|
BIT(1) |
bool |
|
BIT(n > 1) |
binary(n) |
|
BLOB, MEDIUMBLOB, LONGBLOB |
Not Supported |
|
BOOL, BOOLEAN |
int8 |
|
CHAR(n), NCHAR(n), 1≤n≤5000 |
string(n) |
|
CHAR(n), NCHAR(n), n > 5000 |
Not Supported |
|
DATE, YEAR |
date |
|
DECIMAL (unknown precision/scale) |
decfloat34 |
|
DECIMAL(p,s) |
decimal(p,s) |
|
DOUBLE, REAL |
float64 |
|
ENUM, SET |
Not Supported |
|
FLOAT |
float32 |
|
GEOMETRY, GEOMETRYCOLLECTION, LINESTRING, MULTILINESTRING, MULTIPOINT, MULTIPOLYGON, POINT, POLYGON |
Not Supported |
|
INT, MEDIUMINT |
int32 |
|
INT UNSIGNED, MEDIUMINT UNSIGNED |
uint64 |
|
JSON |
Not Supported |
|
SMALLINT |
int16 |
|
SMALLINT UNSIGNED |
uint64 |
|
TEXT, MEDIUMTEXT, LONGTEXT, 1≤n≤5000 |
string(n) |
|
TEXT, MEDIUMTEXT, LONGTEXT, n > 5000 |
Not Supported |
|
TIME |
time |
|
TIMESTAMP(n), DATETIME(n) |
timestamp |
|
TINYBLOB |
binary(n) |
|
TINYINT |
int8 |
|
TINYINT UNSIGNED |
uint8 |
|
TINYTEXT |
string(n) |
|
VARCHAR(n), NVARCHAR(n), 1≤n≤5000 |
string(n) |
|
VARCHAR(n), NVARCHAR(n), n > 5000 |
Not Supported |