Kafka Engine

Last updated:

ClickHouse provides a table engine called Kafka Engine to consume Kafka messages, convert the messages into table rows, and persist the table rows into the destination table. Although a Kafka engine can be configured with multiple topics, a Kafka engine can only have one table schema defined.

PostHog uses the Kafka engine type for several tables:

clickhouse-client --query "SELECT name FROM system.tables WHERE database = 'posthog' AND engine = 'Kafka'"

Note: SELECT is not particularly useful for reading messages (except for debugging), because each message can be read only once. You will lose data if you manually read from this table, as you’ll artificially increment the internal offset ClickHouse uses to read from this table.


Was this page useful?

Next article

Resize disk

How-to Connect to the Clickhouse container to verify the data directory filesystem size (in this example 20G) In your Helm chart configuration, update the clickhouse.persistence.size value in value.yaml to the target size (40G in this example) Run a helm upgrade Connect to the ClickHouse container to verify the new filesystem size Troubleshooting If the resize didn't work check for errors in the ClickHouse operator pod. This procedure doesn't work to decrease a volume. If you try, the disk…

Read next article