Description
Describe the bug
In dbt_utils<0.7.4
, while using get_column_values
macro, dbt compile
on a dummy schema that doesn't exist in the database yet, returns the default
values specified in the macro.
But, In dbt_utils==0.7.4
, dbt compile
is trying to query the column values even though the table does not exist yet and throws schema <dummy_schema_1111> does not exist
error.
Steps to reproduce
Step 1:
This is a sample model that we have.
{{ config(materialized='view', enabled=true) }}
{%- set activity_relation= ref('stg_sf__activities') -%}
with pivot_activity_type as (
select
sf_account_id,
reporting_date,
{{
dbt_utils.pivot(
'activity_type',
dbt_utils.get_column_values(
activity_relation,
'activity_type',
default=['email', 'call']
),
prefix= 'activity_type_'
)
}}
from {{activity_relation}}
group by sf_account_id, reporting_date
)
select * from pivot_activity_type
Step 2:
Set schema
in profiles.yml
to a dummy value.
Step 3:
run dbt compile
Expected results
Previously the macro has:
{%- set target_relation = adapter.get_relation(database=table.database,
schema=table.schema,
identifier=table.identifier) -%}
which sets target_relation
to null when it doesn't exist and the code return default values ['email', 'call']
and the compilation succeeds without querying the database.
Actual results
Now since the code is changed to:
{%- set target_relation = table -%}
it returns schema does not exist
error.
System information
The contents of your packages.yml
file:
packages:
- package: dbt-labs/dbt_utils
version: 0.7.4
Which database are you using dbt with?
- redshift
The output of dbt --version
:
installed version: 0.21.0
latest version: 0.21.0
Up to date!
Plugins:
- redshift: 0.21.0
- postgres: 0.21.0
Additional context
The code doesn't go to the below block even if the relation doesn't exist in the database:
{%- elif not target_relation and default is not none -%}
{{ log("Relation " ~ table ~ " does not exist. Returning the default value: " ~ default) }}
{{ return(default) }}
Are you interested in contributing the fix?
Yes, I would love to contribute.
Activity