Message Flow

Adapter message flow is initiated by the start command.

The Data Source Handler receives real-time market data messages in MAMA format. It receives order book recaps and updates, such as deltas and complex deltas, in MAMDA format. It resolves the subscription key and routes the message to the appropriate data stream, where the data fields extract and convert into stream records.

A record is then ready to be published to Event Stream Processor but is not published immediately. Records are queued, then picked up by SpPublisher. Set the queue capacity in the adapter configuration file. A larger queue is less likely to overflow if a message burst occurs. When the queue is three-quarters full, a warning is logged. Another warning is logged when the queue returns to three-quarters empty. If the queue is full, the adapter waits until room becomes available before it places the next record.

The adapter uses record pooling for performance considerations. The dequeued records are pooled according to user preferences in the adapter configuration file. Messages are published when the pool size reaches the Maximum Pool Size or after a Maximum Pooling Time since the previous publication, whichever occurs first. The Maximum Pooling Time drives the adapter's latency. If the Maximum Pooling Time is too short or Maximum Pool Size is too small, messages are published to Event Stream Processor in undersized batches, resulting in poor overall performance.

When a pooled record batch is ready for publication, the SpPublisher uses the Pub/Sub API to send the records to Event Stream Processor. Records are published asynchronously. The adapter receives no feedback from Event Stream Processor. In the event of a failover, the Pub/Sub API switches to the spare Event Stream Processor instance without message loss.