Example end to end data engineering project.

Replicate data from MySQL, Postgres and MongoDB to ClickHouse®

For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR

Explore Apache Kafka data pipelines in Kubernetes.

🏥 A simple healthcheck wrapper to monitor Kafka Connect.

🎼 Docker compose files for various kafka stacks

A real-time event pipeline around Kafka Ecosystem for Chicago Transit Authority.

Udacity Data Streaming Nanodegree Program

Playground for Kafka / Confluent Kubernetes experimentations

A simple demo showing how to use Ably and fastAPI to route messages into Kafka for stream processing

Streaming Data from Kafka to Postgres with Kafka Connect, AVRO, Schema Registry and Python

Deploy Kafka pipelines to Kubernetes

Web UI for Apache Kafka Connect (vue-connect)

Big data technologies

Store events and publish to Kafka

Guardian for your Kafka Connect connectors. It check status of connectors and tasks and restart if they are failed

Stream CDC into an Amazon S3 data lake in Apache Iceberg format with AWS Glue Streaming using Amazon MSK and MSK Connect (Debezium)

This project is about database replication using Apache Kafka in near real-time, from Postgres to MS SQL Server.

My project submission for Udacity Data Streaming Nanodegree

A tool to manage Apache Kafka Connect connectors and tasks