End-to-end Azure data engineering pipeline ingesting real-time earthquake data from the USGS API. Implements a Bronze–Silver–Gold lakehouse using Azure Data Factory, Databricks, ADLS Gen2, and Synapse Analytics, with both manual execution and fully automated daily-triggered workflows.

End-to-end Azure Data Engineering project using Medallion Architecture, Databricks, Synapse, and Power BI.

Kafka Connect Sink for ADLS Gen2 with SAS auth, Avro support, GZIP compression, and partition-based file batching.

GitOps-driven Azure Data Factory pipeline that ingests multi-source data (GitHub + ADLS) into ADLS Bronze using dynamic, parameterized ETL workflows.

End-to-end Azure Data Engineering pipeline using ADF, Databricks (PySpark), ADLS Gen2, Azure SQL, and Power BI for COVID-19 analytics

In this project we will orchestrate the movement of raw nyc taxi data from staging layer to presentation layer and to semantic model for creating visualizations.

Real-time streaming data pipeline using Apache Kafka, Spark Structured Streaming, and Delta Lake on Azure. Secure SSL Kafka integration, ADLS storage with OAuth2, and ML-driven anomaly detection with automated email alerts. Modular, scalable, and configurable for IoT and log analytics pipelines.

End-to-end data engineering pipeline implementing Medallion Architecture (Bronze-Silver-Gold) for trip transaction analytics. Automated ETL using Azure Data Factory, Databricks, and Delta Lake with real-time monitoring and email notifications via Logic Apps.

ETL/ELT of Grocery Sales data by Azure Data Factory (ADF)

🚀📊 This project demonstrates a complete ETL pipeline for retail sales data using ☁️ Azure Databricks, 📦 ADLS Gen2, and PySpark, following the 🟫 Bronze → 🟦 Silver → 🟨 Gold architecture.

🚀 Build an automated data pipeline with Azure Data Factory and Databricks to efficiently process and analyze trip transaction data using Medallion Architecture.

🌟 Write records from Kafka to Azure Data Lake Storage Gen2 with this efficient sink connector using SAS authentication and optional GZIP compression.