Gem used to send messages to Kafka in an easy way with an extra validation layer. It is a part of the Karafka ecosystem.

WaterDrop is based on Zendesks delivery_boy gem.

It is:

• Thread safe
• Supports sync and async producers
• Working with 0.11+ Kafka

gem install waterdrop

or add this to your Gemfile:

gem 'waterdrop'

and run

bundle install

WaterDrop is a complex tool, that contains multiple configuration options. To keep everything organized, all the configuration options were divided into two groups:

• WaterDrop options - options directly related to Karafka framework and it's components
• Ruby-Kafka driver options - options related to Ruby-Kafka/Delivery boy

To apply all those configuration options, you need to use the #setup method:

WaterDrop.setup do |config|
  config.deliver = true
  config.kafka.seed_brokers = %w[kafka://localhost:9092]
end

Note: We've listed here only the most important configuration options. If you're interested in all the options, please go to the config.rb file for more details.

Note: All the options are subject to validations. In order to check what is and what is not acceptable, please go to the config.rb validation schema file.

This configuration can be also placed in config/initializers and can vary based on the environment:

WaterDrop.setup do |config|
  config.deliver = Rails.env.production?
  config.kafka.seed_brokers = [Rails.env.production? ? 'kafka://prod-host:9091' : 'kafka://localhost:9092']
end

To send Kafka messages, just use one of the producers:

WaterDrop::SyncProducer.call('message', topic: 'my-topic')
# or for async
WaterDrop::AsyncProducer.call('message', topic: 'my-topic')

Both SyncProducer and AsyncProducer accept following options:

Keep in mind, that message you want to send should be either binary or stringified (to_s, to_json, etc).

• Karafka framework
• WaterDrop Travis CI
• WaterDrop Coditsu

First, thank you for considering contributing to the Karafka ecosystem! It's people like you that make the open source community such a great community!

Each pull request must pass all the RSpec specs, integration tests and meet our quality requirements.

Fork it, update and wait for the Github Actions results.