Reader
Read messages from Pulsar topic(s) without subscription.
type: "io.kestra.plugin.pulsar.Reader"
id: pulsar_reader
namespace: company.team
tasks:
- id: reader
type: io.kestra.plugin.pulsar.Reader
uri: pulsar://localhost:26650
topic: test_kestra
deserializer: JSON
YES
STRING
STRING
JSON
BYTES
Deserializer used for the value.
YES
Pulsar topic(s) where to consume messages from.
Can be a string or a list of strings to consume from multiple topics.
YES
Connection URLs.
You need to specify a Pulsar protocol URL.
- Example of localhost:
pulsar://localhost: 6650
- If you have multiple brokers:
pulsar://localhost: 6650,localhost: 6651,localhost: 6652
- If you use TLS authentication:
pulsar+ssl://pulsar.us-west.example.com: 6651
YES
Authentication token.
Authentication token that can be required by some providers such as Clever Cloud.
YES
duration
The maximum duration waiting for new record.
It's not a hard limit and is evaluated every second.
YES
The maximum number of records to fetch before stopping.
It's not a hard limit and is evaluated every second.
YES
Position the reader on a particular message.
The first message read will be the one immediately after the specified message.
If no since
or messageId
are provided, we start at the beginning of the topic.
YES
PT2S
duration
Duration waiting for record to be polled.
If no records are available, the maximum wait to wait for a new record.
YES
JSON string of the topic's schema
Required for connecting with topics with a defined schema and strict schema checking
YES
NONE
NONE
AVRO
JSON
The schema type of the topic
Can be one of NONE, AVRO or JSON. None means there will be no schema enforced.
YES
duration
The initial reader positioning can be set at specific timestamp by providing total rollback duration.
So, broker can find a latest message that was published before given duration. eg: since
set to 5 minutes (PT5M
) indicates that broker should find message published 5 minutes in the past, and set the initial position to that messageId.
NO
TLS authentication options.
You need to use "pulsar+ssl://" in serviceUrl to enable TLS support.
Number of messages consumed.
uri
URI of a Kestra internal storage file containing the consumed messages.
YES
The ca certificate.
Must be a base64-encoded pem file.
YES
The client certificate.
Must be a base64-encoded pem file.
YES
The key certificate.
Must be a base64-encoded pem file.