Google Pub/Sub Consumer
This operator consumes publications from a Google Pub/Sub topic and outputs them as a stream of messages.
-
If Subscription name refers to an existing subscription name on the Google Pub/Sub service, Topic can be empty, as an existing subscription is already attached to a topic. If subscription is to be created, Topic must be provided and Topic must exist on the pub/sub service. Otherwise, the graph fails regardless of the value of Fail on error.
-
If Fail on error is false, upon error, the consumer only outputs the error message on Error output port and does not fail the graph.
-
The messageToAck input port is optional. If connected, no automatic acknowledgment is sent and the user has to explicitly acknowledge received publications by submitting their message ID to this port. This allows using the Google Pub/Sub service as a message queue and ensures that publications that are not consumed successfully get delivered again to the same consumer or other consumers using the same Subscription name.
-
When using explicit acknowledgment, receiving more than one ack for the same publication or requesting acknowledgment for a publication that is not received by the consumer, results in an error.
Configuration Parameters
Parameter |
Type |
Description |
---|---|---|
Connection |
string |
Mandatory: A Google Pub/Sub connection consisting of a GCP project ID and a JSON key file to access the Pub/Sub service. |
Subscription name |
string |
Mandatory: The name of the subscription on the Google Pub/Sub service. |
Topic |
string |
Google Pub/Sub topic name. |
Fail on error |
boolean |
Whether to fail the whole graph if the consumer encounters an
error at runtime.
Default: false |
Maximum acknowledgment time in seconds | integer | Maximum time period between when the consumer receives the
message and it sends the acknowledgment to the Google Pub/Sub
service. If not acknowledged within this period, the Pub/Sub service
delivers the publication again to the same consumer or any other
consumer which is configure to use the same Subscription name. If
the messageToAck input port is not used, the consumer automatically
sends the acknowledgment for a message upon receiving it from the
pub/sub service.
Default: 600 |
Maximum outstanding publications | integer | Maximum number of received but unacknowledged messages on the
consumer. Upon reaching this maximum, the consumer stops pulling
publications form the pub/sub service until one or more of the
existing messages expire (ack deadline is passed) or acknowledged. A
negative value indicates no limit for the number of unprocessed
messages.
Default: 1000 |
Input
Input |
Type |
Description |
---|---|---|
messageToAck |
message |
The stream of Message IDs. Each publication received by the consumer has a unique ID assigned by the Google Pub/Sub service. Using this ID, the publication can be explicitly acknowledged. The operator reads the message ID from the attribute named msg.MessageID in the received message. |
Output
Output |
Type |
Description |
---|---|---|
receivedMessage |
message |
The stream of publications received by the consumer from the
Topic on Google Pub/Sub service. Each output message contains the
following extra two attributes:
|
error |
message |
Error messages occurring during runtime are output on this port as a stream of messages (one for each error). The error message has the attribute msg.KeyError with the value true and the body of the message is the error message. |