Implementation of Spark code in Jupyter notebook. Topics include: RDDs and DataFrame, exploratory data analysis (EDA), handling multiple DataFrames, visualization, Machine Learning

A big data project to develop a real-time data pipeline for analyzing the popularity and sentiments of trending topics on Twitter.

Getting started with PySpark for Big data analysis

Efficiently tackle large datasets and perform big data analysis with Spark and Python

MapReduce Job Development, RDDs Programming, Medical Data Management, Sales Analysis, And Efficient Data Integration For Big Data Analysis. Spark: Big Data Processing, SQOOP Integration, And Spark Structured Streaming For Real-Time Data.

Pyspark studies.

Data Mining using Spark Rdds

Spark, RDDs and Map Reduce applications related to the BigData @polito course (2019-2020). A set of personal notes are already provided.

This assignment was part of an IoT motion sensor App running on a watch, predicting actions of the individual wearing the watch based on his arm movements; this IoT Analytics assignments is one of a series of data pipeline coding challenges in the IBM course Scalable Data Science.

This project illustrates Apache Spark RDD operations, from creation and transformation to actions and results, enhancing users' understanding of distributed data processing.

Analysis of Clinical Trial Dataset using PySpark RDD implementation.

Project on MapReduce for the Μ111 - Big Data Management course, NKUA, Spring 2023.

Here I play with the services offered by Apache Spark and try to learn them in more depth.

📈📊 Big Data Notebooks . ▫️ Análisis masivos de datos con pyspark ▫️ Ingesta de datos. ▫️ Algoritmos de machine learning con datos masivos. ▫️ Procesamiento de mensajes en tiempo real con Kafka.