-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Description
Describe the bug
When using the below recipe template (in which the metadata paths are working as expected and sink connection is established), I receive an error message as PATH/Python312\site-packages\datahub\ingestion\run\pipeline.py", line 140, in _add_init_error_context
raise PipelineInitError(f"Failed to {step}: {e}") from e
datahub.ingestion.run.pipeline.PipelineInitError: Failed to find a registered source for type dbt: dbt is disabled due to an error in initialization
pipeline_name: "[PIPELINE_NAME]"
source:
type: "dbt"
config:
# Coordinates
manifest_path: "[PATH_TO_MANIFEST.JSON]" #MANIFEST metadata
catalog_path: "[PATH_TO_CATALOG.JSON]" #CATALOG metadata
#sources_path: "[PATH_TO_SOURCES.JSON]" #SOURCES metadata
test_results_path: "[PATH_TO_RESULTS.JSON]" #RESULTS metadata
#only_include_if_in_catalog: true
# Options
target_platform: "bigquery" # e.g. bigquery/postgres/etc.
stateful_ingestion:
enabled: true
remove_stale_metadata: false
node_name_pattern: # allows for inclusion or exclusion of assets for the ingestion via regular expressions
ignoreCase: true
allow:
- .*[]
deny:
- .*[]
#Setup connection to datahub env
sink:
type: "datahub-rest"
config:
server: "[LOCALHOST]"
token: "[ACCESS TOKEN]"
To Reproduce
Steps to reproduce the behavior:
- Using virtual environment with wheel
- Using DataHub CLI version: 0.14.1.6
- 'acryl-datahub[dbt]' is installed
- Run ingestion as: datahub ingest -c [PATH]\dbt_meta_ingest.yaml
Expected behavior
Ingestion Pipeline finished
Desktop (please complete the following information):
- OS: WINDOWS 10
- Command line: Windows powershell
Additional context
Add any other context about the problem here.
