What does gpkafka do?
gpkafka is a Greenplum utility used for integrating Greenplum Database with Apache Kafka. It allows you to connect to Kafka topics, consume messages, and load data into your Greenplum Database for analysis and processing.
Why is it important?
Integrating Greenplum with Kafka is essential for real-time data processing and analytics. It enables you to ingest streaming data from Kafka into Greenplum, making it available for immediate analysis and reporting.
How to use gpkafka:
You can use gpkafka to connect Greenplum with Kafka. Here's how to use it:
Example: gpkafka --topic my_topic --broker my_broker --group my_group --database my_db --table my_table
gpkafka --topic topic_name
--broker kafka_broker
--group consumer_group
--database database_name
--table target_table
[ --start start_offset ]
[ --end end_offset ]
[ --batch-size batch_size ]
[ --max-poll-records max_records ]
[ --poll-timeout timeout_ms ]
[ --checkpoint-interval interval_ms ]
[ --separator field_separator ]
[ --ssl ]
[ --ssl-ca-location ca_certificate_path ]
[ --ssl-cert-location client_certificate_path ]
[ --ssl-key-location client_key_path ]
[ --ssl-key-password key_password ]
[ --help ]
[ --version ]