Avro Ingest Mod via Disk
From a Kafka broker, a series of messages are read, each consisting of Avro records. Each message is passed to the preingestor, which creates a message with record objects. Finally, this message is passed to the ingestor, which stores the records in the Vora disk engine and emits the commit token associated with each message.
- You need a running Vora instance and a Kafka broker.
Configure and Run the Graph
- In the left panel, select the Graphs tab and navigate to com/sap/demo/vora/ingestion/avro_ingestion_example_disk_go.
- Check the configuration of the consumer2 node: brokers and topics
- Check the configuration of the ingestor node: dsn
- In the tool bar, select Run (play button).
- The Status panel indicates if the graph is running.
- Use the context menu Open UI of the Wiretap node to open the wiretap.
- Send Avro messages to the specified Kafka topic.
- The wiretap opens and you see the commit tokens.
- Stop the graph and change the generator’s batchSize and run the graph again.