🎉 Checkout the brand new Kattlo Documentation site 🎉

• https://kattlo.github.io

Use an approach like Database Migrations to manage your evolutionary configurations for:

• Topics
• Schemas
• ACLs
• ksqlDB
• Connect
• Cluster
• Quotas
• Users

💡 Check the examples directory 💡

• enterprises that needs a stable way to change Apache Kafka® configurations
• maintaining the configuration and avoid drifts
• knowing when a topic was removed (when its managed by Kattlo)
• accessing the history of migrations
• your DevOps toolset to properly manages the topic within the clusters

• Java 11
• Picocli
• Quarkus
• GraalVM
• and a lot of coffee ☕

• See installation instructions for MacOS and Windows

curl 'https://github.com/kattlo/kattlo-cli/releases/download/v0.2.2/kattlo-v0.2.2-linux' \
  -o 'kattlo'

sudo chmod +x kattlo
sudo mv kattlo /usr/local/sbin/kattlo

The are .deb and .rpm packages available. Do a check in the latest release.

• Topic migrations
  • apply migrations
  • import existing topics
  • show info and history
  • generate migration example
  • rules enforcement ✨
  • human readable values for configurations ✨
• ACL migrations
• Schema migrations
• Quota migrations
• User migrations
• Connect migrations
• ksqlDB migrations
• Cluster migrations
• Utilities
  • init project
  • new config for consumers
  • new config for producers

• --config-file (optional): Path to Kattlo configuration file for migrations
• --kafka-config-file (required): Path to properties file to be used for Kafka Admin Client

In the .kattlo.yaml configuration file you may define the following properties:

rules:
  topic:
    namePattern: 'your pattern'
    # more rules constraints...

In the --kafka-config-file you may put the properties described at official documentation.

Example of kafka.properties:

bootstrap.servers=localhost:19092,localhost:29092
client.id=kattlo-cli

Command:

kattlo --config-file='.kattlo.yaml' \
       --kafka-config-file='kafka.properties' \
       <command>
       [command arguments]

You can call for help for any command:

kattlo -h

# topic
kattlo topic -h

# init
kattlo init -h

To make easy the process to write down the migrations, you may use then gen command to genereate migration files:

kattlo gen migration \
  --resource=TOPIC \
  --diretory='/path/to/gen/migration'

If you suppress the --directory option, the migration example will be gerenated in the current directory.

Kattlo provide a way to declare what we want using yaml notation. Based on that files, Kattlo runs the necessary Admin commands to create, patch or remove resources.

See examples to see the variations of usage.

Resources can be:

• topics
• schemas
• ACLs

See Kattlo's docs about Topics Migrations

Kattlo needs to have all permissions to manage topics, ACLs and Schemas configurations, outherwise you will be not able to perform the migrations.

In order to manage the migrations we use special topics:

• __kattlo-topics-state: the topics' migrations state
• __kattlo-topics-history: the topics' migrations history

To persist the current state per topic.

This topic has the following configurations:

• partitions: 50
• desired replication-factor: 2

To persist the histories for topics.

This topic has the following configurations:

• partitions: 50
• desired replication-factor: 2

./gradlew clean build -Dquarkus.package.type=native \
 -Dquarkus.native.container-build=true \
 -Dquarkus.native.additional-build-args=--report-unsupported-elements-at-runtime,--allow-incomplete-classpath,-H:IncludeResources='.*yaml$',-H:IncludeResources='.*json$',-H:Log=registerResource:

You can then execute your native executable with: ./build/kattlo-1.0-SNAPSHOT-runner

Configure -Dquarkus.native.container-build to false if you want o use your graalvm installation instead of Docker image.

If you are experiencing issues, run Kattlo with logging set to DEBUG.

kattlo -Dquarkus.log.level=DEBUG ...

The application can be packaged using ./gradlew quarkusBuild. It produces the kattlo-1.0-SNAPSHOT-runner.jar file in the build directory. Be aware that it’s not an über-jar as the dependencies are copied into the build/lib directory.

The application is now runnable using java -jar build/kattlo-1.0-SNAPSHOT-runner.jar.

If you want to build an über-jar, just add the --uber-jar option to the command line:

./gradlew quarkusBuild --uber-jar

• Kattlo icon by Freepik from www.flaticon.com