Scalable Data Version Control

Manage your data as code using Git-like operations and achieve reproducible, high-quality data pipelines. Available Open Source or on the Cloud.

Take control of your data

Use Cases

lakeFS helps data engineers and data scientists in every field manage their data like code — at scale

Data Science
Data engineering
Data
Ops

Data Collaboration
Provide your team with tools to easily collaborate and communicate on the data they use. Utilizing Git-like semantics, share a branch of a data repository or a commit ID to specify the data version being used or shared.

Data Auditing
Keep track of the data changes made, and by whom.
A full audit on all data-related actions, in all environments, allow you to trace back any result provided or experiment performed.
Read more →

Storage Cost Reduction
Prevent your data lake from becoming a data swamp.
The use of a zero-copy branch allows data practitioners to get an isolated data lake for their use without creating actual copies that increase costs and pollute the data lake.
Read more →

lakeFS is already helping thousands of organizations

Here's what ML and Data Engineers using lakeFS have to say

lakeFS saved us from the analysis paralysis of overthinking how to test new software on our data lake at Netflix scale. In less than 20 min I had lakeFS up and running, and was able to run tests against my production data in isolation and validate the software change thoroughly before pushing to production. With lakeFS, we improved the robustness and flexibility of our data systems.

Holden Karau,
Open Source Engineer

Moving to a data branching solution has paid off quickly for us. A few days after completing the migration, we’ve already reduced testing time by 80% on two different projects. And we’re excited to see how data branching increases our product velocity.

Ryan Green,
CTO

The cloud never warned us about the data getting clouded. As the blessing of infinite storage quickly became an unmanageable mess, there is a need for technologies like lakeFS to make data accessible again

Sivan Bercovici,
CTO

With lakeFS we can easily achieve advanced use cases with data, such as running parallel pipelines with different logic to experiment or conduct what-if analysis, compare large result sets for data science and machine learning, and more

Stephen Seewald,
Raghvendra Verma,
Cory Matheson

Since introducing lakeFS to our production data environment, we’ve enjoyed the benefits of atomic and isolated operations in our data pipelines. This has allowed us to spend more time improving other aspects of our data platform, and less time dealing with the fallout from race conditions and partially failed operations

Lior Resisi,
Data Platform Team Lead

By using lakeFS we produce a commit history on the production branch that easily allows for rollbacks. In the case of data quality issues in production, this allows us to simply revert to the previous high quality snapshot of our data.

Scalable Data Version Control

Manage your data as code using Git-like operations and achieve reproducible, high-quality data pipelines. Available Open Source or on the Cloud.

Take control of your data

COMPUTE ENGINES

lakefs

formats

Object Storage

Use Cases

Perform Local Checkouts on Data

Deduplicated Experimentation

Reproducible Feature Engineering & Model Training

Isolated Dev/Test Environments

Promote Only High Quality Data to Production

Fix Bad Data with Production Rollback

Data Collaboration

Data Auditing

Storage Cost Reduction

lakeFS is already helping thousands of organizations

Faster time to market

Increased data quality

Improved security and governance

Here's what ML and Data Engineers using lakeFS have to say

Official partners

lakeFS connects to every object storage that uses the S3 interface

lakeFS supports all broadly used compute engines

All common ingest technologies are integrated into lakeFS

lakeFS is format agnostic! Regardless of the format you’re using, lakeFS will support it

Open table

Data Formats

Unstructured

Manage Orchestration and Workflows better with popular orchestration tools supported on lakeFS

Seamlessly integrate with lakeFS, with whatever your ML stack includes

Data Quality is mandatory for your data lake health. Ensure/maintain the highest data quality together with lakeFS

lakeFS Data Version Control Blog

lakeFS and dbt: A Modern Way to Manage Your Data

AI in Data Engineering: Challenges, Best Practices & Tools

Compliance in the Age of LLMs: The Role of Data Versioning

Need help getting started?

Git for Data – lakeFS

Pick up the Slack with lakeFS

Faster time to
market

Increased data
quality

Improved security
and governance