Skip to content

"if not target_relation" is not true even if relation does not exist in get_column_values #444

Closed
@mothiki

Description

Describe the bug

In dbt_utils<0.7.4, while using get_column_values macro, dbt compile on a dummy schema that doesn't exist in the database yet, returns the default values specified in the macro.
But, In dbt_utils==0.7.4, dbt compile is trying to query the column values even though the table does not exist yet and throws schema <dummy_schema_1111> does not exist error.

Steps to reproduce

Step 1:
This is a sample model that we have.

{{ config(materialized='view', enabled=true) }}

{%- set activity_relation= ref('stg_sf__activities') -%}

with pivot_activity_type as (

  select
    sf_account_id,
    reporting_date,
    {{
      dbt_utils.pivot(
        'activity_type',
        dbt_utils.get_column_values(
          activity_relation,
          'activity_type',
          default=['email', 'call']
        ),
      prefix= 'activity_type_'
      )
    }}

    from {{activity_relation}}

    group by sf_account_id, reporting_date
)

select * from pivot_activity_type

Step 2:
Set schema in profiles.yml to a dummy value.

Step 3:
run dbt compile

Expected results

Previously the macro has:

{%- set target_relation = adapter.get_relation(database=table.database,
                                          schema=table.schema,
                                         identifier=table.identifier) -%} 

which sets target_relation to null when it doesn't exist and the code return default values ['email', 'call'] and the compilation succeeds without querying the database.

Actual results

Now since the code is changed to:

{%- set target_relation = table -%}

it returns schema does not exist error.

System information

The contents of your packages.yml file:

packages:
  - package: dbt-labs/dbt_utils
     version: 0.7.4

Which database are you using dbt with?

  • redshift

The output of dbt --version:

installed version: 0.21.0
  latest version: 0.21.0

Up to date!

Plugins:
  - redshift: 0.21.0
  - postgres: 0.21.0

Additional context

The code doesn't go to the below block even if the relation doesn't exist in the database:

        {%- elif not target_relation and default is not none -%}

          {{ log("Relation " ~ table ~ " does not exist. Returning the default value: " ~ default) }}

          {{ return(default) }}

Are you interested in contributing the fix?

Yes, I would love to contribute.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpending

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions