Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

How to implement a streaming at scale solution in Azure

Apache Spark Connector for Azure Cosmos DB

Basic logging mechanism for Node.js Lambda Functions and other AWS services, which produces logs in JSON format for easier reading through Cloudwatch Logs.

A full big data pipeline (Lambda Architecture) with Spark, Kafka, HDFS and Cassandra.

Canari v3 - next gen Maltego framework for rapid remote and local transform development

📝 A step-by-step example/tutorial showing how to build a Phoenix (Elixir) App where all data is immutable (append only). Precursor to Blockchain, IPFS or Solid!

Big Data webapp using Chicago street congestion, crashes, red light violations, and speed camera violations

Nuxt.js Serverless SSR using Cloudfront, Lambda@edge and S3

Developing a Lambda Architecture pipeline using Apache Kafka, Spark Structured Streaming, Redshift, S3, Python

A K8s-based infrastructure for analytics

Serverless Typescript Template

This project aims to predict smartphone prices using a combination of batch and stream processing techniques in a Big Data environment. The architecture follows the Lambda Architecture pattern, providing both real-time and batch processing capabilities to users.

Create data pipeline using Lambda architecture with Spark, Kafka, Airflow and Snowflake

Lambda architecture implementation using Apache Storm, Hadoop and HBase to perform Twitter real-time sentiment analysis

Apache Spark Connector for Azure Cosmos DB

From Fast to Smart Data - Lambda Architecture with Apache Spark, Kafka and Cassandra

A curated list of resources to navigate data engineering

A project which includes simulation of real time queries by kafka and performing stream and batch processing of the simulated queries by spark. Also, this follows lambda architecture, in which kafka is publisher and spark helps in subscribing