-
Notifications
You must be signed in to change notification settings - Fork 3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(ingest): adds experimental support for ingesting Looker metadata (…
- Loading branch information
1 parent
9d00e8b
commit fce8a53
Showing
6 changed files
with
801 additions
and
0 deletions.
There are no files selected for viewing
15 changes: 15 additions & 0 deletions
15
contrib/metadata-ingestion/python/looker/dashboard_ingestion/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
## looker_dashboard_ingestion.py | ||
This tool helps ingest Looker dashboard and chart metadata into datahub. | ||
Currently it creates a separate platform named "looker" and loads all dashboard and chart information into that platform as virtual datasets. This was to workaround datahub's lack of support for dashboard entities, however datahub recently started supporting proper dashboard entities. | ||
|
||
The script assumes you already have run lookml_ingestion.py to scrape view definitions into datahub, this is important because we assign lineage between looker views and looker dashboards/charts where possible. | ||
|
||
|
||
## Steps: | ||
- Use a version of python >= 3.7 | ||
- Make a virtual environment | ||
- pip install -r requirements.txt | ||
- Set env vars: LOOKERSDK_CLIENT_ID, LOOKERSDK_CLIENT_SECRET, LOOKERSDK_BASE_URL | ||
- Configure extra kafka conf in looker_dashboard_ingestion.py | ||
|
||
python looker_dashboard_ingestion.py |
Oops, something went wrong.