Quickstart | Documentation | Features | Roadmap | Adoption | FAQ | Town Hall

📣 Next DataHub town hall meeting on July 31st, 9am-10am PDT:

• Signup sheet & questions
• Details and recordings of past meetings can be found here

✨ Latest Update:

• We've released v0.4.1. You can find release notes here
• We're on Slack now! Join or log in with an existing account. Ask questions and keep up with the latest announcements.

DataHub is LinkedIn's generalized metadata search & discovery tool. To learn more about DataHub, check out our LinkedIn blog post and Strata presentation. You should also visit DataHub Architecture to get a better understanding of how DataHub is implemented and DataHub Onboarding Guide to understand how to extend DataHub for your own use case.

This repository contains the complete source code for both DataHub's frontend & backend. You can also read about how we sync the changes between our internal fork and GitHub.

1. Install docker and docker-compose (if using Linux). Make sure to allocate enough hardware resources for Docker engine. Tested & confirmed config: 2 CPUs, 8GB RAM, 2GB Swap area.
2. Open Docker either from the command line or the desktop app and ensure it is up and running.
3. Clone this repo and cd into the root directory of the cloned repository.
4. Run the following command to download and run all Docker containers locally:

   ./docker/quickstart/quickstart.sh
   

   This step takes a while to run the first time, and it may be difficult to tell if DataHub is fully up and running from the combined log. Please use this guide to verify that each container is running correctly.
5. At this point, you should be able to start DataHub by opening http://localhost:9001 in your browser. You can sign in using datahub as both username and password. However, you'll notice that no data has been ingested yet.
6. To ingest provided sample data to DataHub, switch to a new terminal window, cd into the cloned datahub repo, and run the following command:

   ./docker/ingestion/ingestion.sh
   

   After running this, you should be able to see and search sample datasets in DataHub.

Please refer to the debugging guide if you encounter any issues during the quickstart.

• DataHub Developer's Guide
• DataHub Architecture
• DataHub Onboarding Guide
• Docker Images
• Frontend
• Web App
• Generalized Metadata Service
• Metadata Ingestion
• Metadata Processing Jobs

See Releases page for more details. We follow the SemVer Specification when versioning the releases and adopt the Keep a Changelog convention for the changelog format.

Frequently Asked Questions about DataHub can be found here.

Check out DataHub's Features & Roadmap.

We welcome contributions from the community. Please refer to our Contributing Guidelines for more details. We also have a contrib directory for incubating experimental features.

Join our slack workspace for discussions and important announcements. You can also find out more about our past and upcoming town hall meetings.

Here are the companies that have officially adopted DataHub. Please feel free to add yours to the list if we missed it.

• Expedia Group
• LinkedIn
• Saxo Bank
• Shanghai HuaRui Bank
• TypeForm
• Valassis

Here is a list of companies currently building POC or seriously evaluating DataHub.

• Booking.com
• Experian
• Geotab
• Instructure
• Microsoft
• Morgan Stanley
• Orange Telecom
• SpotHero
• Sysco AS
• ThoughtWorks
• University of Phoenix
• Vectice

• DataHub: A Generalized Metadata Search & Discovery Tool
• Open sourcing DataHub: LinkedIn’s metadata search and discovery platform
• The evolution of metadata: LinkedIn’s story @ Strata Data Conference 2019
• Journey of metadata at LinkedIn @ Crunch Data Conference 2019
• DataHub Journey with Expedia Group by Arun Vasudevan
• Data Catalogue — Knowing your data
• LinkedIn DataHub Application Architecture Quick Understanding
• 25 Hot New Data Tools and What They DON’T Do

See the full list here.