avrocat

A CLI tool for consuming events from Kafka, deserializing them with Avro schemas, and printing the output as JSON.

Quick start

# local schema file
$ avrocat -b localhost:9092 -s path/to/schema.avsc -t my-topic

# schema from Confluent Schema Registry
$ avrocat -b localhost:9092 -t my-topic -r http://localhost:8081

Topic can be omitted when using --schema: it is inferred from the schema filename (strips the .avsc extension and leading path).

Usage

Usage: avrocat <command>

Flags:

  -V, --Version          Kafka version (default: 0.8.2.0)
  -b, --broker(s)        Bootstrap broker(s) (host[:port]), comma-separated (default: <none>)
  -d, --debug            enable debug logging (default: false)
  -o, --offset           The offset to start with. Can be `oldest` or `newest` (default: newest)
  -p, --partitions       The partitions to consume, can be 'all' or comma-separated numbers (default: all)
  -s, --schema           Path to avro schema file (mutually exclusive with --registry)
  -r, --registry         Confluent Schema Registry base URL (e.g. http://localhost:8081)
  --registry-subject     Schema Registry subject name (default: {topic}-value)
  -t, --topic            Topic to consume from (default: <none>; required when using --registry)
  --strip-schema-id      Strip the 5-byte Confluent Schema Registry header before decoding (default: true)
  -F, --from             Consume messages from this UTC time, RFC3339 (e.g. 2024-01-01T00:00:00Z); requires --to
  -E, --to               Consume messages up to this UTC time, RFC3339 (e.g. 2024-01-01T01:00:00Z); requires --from
  -S, --sasl             Enable SASL authentication (default: false)
  -U, --username         SASL username
  -P, --password         SASL password
  -M, --mechanism        SASL mechanism: PLAIN (default), SCRAM-SHA-256, SCRAM-SHA-512, OAUTHBEARER, GSSAPI

Commands:

  version  Show the version information.

By default, avrocat expects messages in the Confluent Schema Registry wire format (1 magic byte + 4-byte schema ID prefix). Pass --strip-schema-id=false to consume raw Avro messages without that header.

Schema Registry

Use --registry instead of --schema to fetch the Avro schema at startup from a Confluent Schema Registry. The subject defaults to {topic}-value (the standard naming convention) but can be overridden with --registry-subject.

$ avrocat -b localhost:9092 -t my-topic -r http://localhost:8081

# explicit subject
$ avrocat -b localhost:9092 -t my-topic \
    -r http://localhost:8081 \
    --registry-subject my-topic-value

--schema and --registry are mutually exclusive; exactly one must be provided.

Time-range consumption

Use --from and --to (both required together) to replay only the messages produced within a UTC window. The consumer resolves the corresponding offsets at runtime via the broker and exits automatically once the window is drained — no manual offset management needed.

$ avrocat -b localhost:9092 -s schema.avsc \
    --from 2024-01-15T08:00:00Z \
    --to   2024-01-15T09:00:00Z

When --from/--to are set, the --offset flag is ignored.

SASL authentication

Pass -S to enable SASL. The default mechanism is PLAIN; use -M to select another.

# PLAIN
$ avrocat -b localhost:9092 -s schema.avsc -t my-topic \
    -S -U myuser -P mypassword

# SCRAM-SHA-512
$ avrocat -b localhost:9092 -s schema.avsc -t my-topic \
    -S -M SCRAM-SHA-512 -U myuser -P mypassword

When SASL is enabled, TLS is also enabled automatically.

Docker

Build

It's not necessary to install Golang to run the script, just build the container using docker with the following command:

docker build -t avrocat .

Running

$ docker run --rm -ti --name avrocat \
    --privileged --network=host --pid=host \
    avrocat -b kafka.hmarques.dev.use1.com:9092 \
      -t <topic> \
      -s schema.avsc \
      -d