Kafka Consumer 2
The Kafka Consumer operator works as a Kafka client that consumes records/messages from a Kafka cluster. It is compatible with Kafka versions 0.9.x and 0.10.x. It is not compatible with Kafka version 0.8.x. When using Kafka 0.8.x, use the Kafka Consumer operator, which is compatible with Kafka 0.8.x.
Configuration Parameters
Parameter |
Type |
Description |
---|---|---|
maxWaitTime |
int |
The maximum amount of time in milliseconds that the broker waits for the consumer. Default: 500 |
kafkaVersion |
string |
Mandatory. The version of Kafka installed. Default: "0.9.0" |
brokers |
string |
Mandatory. Each node in the cluster.
Default: "localhost:9092" |
topics |
string |
Mandatory. The category name to which records are published. Default: "test_topic" |
groupId |
string |
Mandatory. Holds all information for a consumer that is part of the group. Default: "test_group" |
offset |
string |
Identifies each record on partition. Default: "newest" |
autoCommit |
bool |
Specifies whether to automatically commit the message. Default: true |
maxMessageBytes |
int |
The maximum permitted size of a message. Default: 1000000 |
useTLS |
bool |
Flag that enables using TLS/SSL authentication. Default: false |
insecureSkipVerify |
bool |
Controls whether a client verifies the server's certificate chain and host name. Default: false |
tlsCAFile |
string |
Certificate authority file path. Default: "" |
authenticate |
bool |
Flag that enables SASL authentication. Default: false |
username |
string |
Username if using kafka's SASL/PLAIN authentication. Default: "" |
password |
string |
Password if using kafka's SASL/PLAIN authentication. Default: "" |
Input
Input |
Type |
Description |
---|---|---|
markOffset |
message |
Marks the provided message as processed. |
Output
Output |
Type |
Description |
---|---|---|
message |
message |
The stream of records produced. |