Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

High-performance Go package to read and write Parquet files

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

Price Crawler - Tracking Price Inflation

💂‍♂️ Tools to Transform and Query Data with 'Apache' 'Drill'

Query and transform data with PRQL

A library for Spark DataFrame using MinIO Select API

MongoDB integrations for Apache Arrow. Export MongoDB documents to numpy array, parquet files, and pandas dataframes in one line of code.

A converter for the OSM PBFs to Parquet files

OSM planet dump high performance data loader. Transform OpenStreetMap World/Region PBF dump into partitioned by H3 regions PostGIS pgsnapshot (lossless) OSM schema representation and/or into ArrowIPC/Parquet dumps

Library to read a subset of Parquet files

A lightweight Java library that facilitates reading and writing Apache Parquet files without Hadoop dependencies

Threat Detection and Visualization

Scala ZIO-powered Apache Parquet library

Converts between file formats such as CSV and Parquet

A web application for viewing Apache Parquet files . This is a Python + Flask application

Squey is a visualization software designed to interactively explore and understand large amounts of tabular data (this is the read-only mirror of https://gitlab.com/squey/squey)

Streaming data of Tiki with Kafka and processing with Spark, visualize with Elasticsearch & Kibana.

‼️ Handle Big Data for Machine Learning using Python and PySpark, Building ETL Pipelines with PySpark, MongoDB, and Bokeh