Skip to content

dbt_utils.get_filtered_columns_in_relation has inconsistent type between parsing and runtime #969

Open
@cvm-a

Description

Describe the bug

dbt_utils.get_filtered_columns_in_relation returns '' during parsing, but returns a list during runtime. This makes it very hard to use this with conditional concatenation or with with filters, since the type changes from parsing and execution.
This results in errors like can only concatenate str (not "list") to str

Steps to reproduce

When creating a macro that takes two relations, one of which is optional, and trying to get all the column names, we may wan tot concatenate the list of column names like

{% macro all_columns (table_ref1, table_ref2=none) -%}
{{
    return (
        dbt_utils.get_filtered_columns_in_relation(table_ref1) +
       (dbt_utils.get_filtered_columns_in_relation(table_ref2) if table_ref2 is not none else [])                            
)
}}
{%- endmacro %}


Expected results

The concatenation should just work -- I should get the columns of both table_ref1 and table_ref2 if table_ref2 is not none, otherwise, I should only get the columns of table_ref1.

Actual results

during parsing, dbt_utils.get_filtered_columns_in_relation(table_ref1) is just '', so we get an error can only concatenate str (not "list") to str. If I change the else to '' from [] then it will raise can only concatenate list (not "str") to list as a Compilation error for the specific model selected.

Screenshots and log output

System information

The contents of your packages.yml file:
packages:

  • package: dbt-labs/dbt_utils
    version: 1.1.1

Which database are you using dbt with?

  • postgres
  • redshift
  • [ *] bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

Core:
  - installed: 1.8.7
  - latest:    1.8.9 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - bigquery: 1.8.3 - Up to date!

Additional context


has return('')

Are you interested in contributing the fix?

sure I would replace that with return([])

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions