Description
Describe the bug
I have configured the equality test on a model, but if that model hasn't been built, then I'm unable to generate docs or run dbt compile
without an error message about the tables not having the same number of columns. However, I don't want to have to build every model that has an equality test on it just to view the docs or compile the project, and would expect it to either defer to the production model, or include placeholder values.
Steps to reproduce
- Create a model and include the equality test against an existing model:
- name: core_users
data_tests:
- dbt_utils.equality:
compare_model: source('comparison_source', 'core_users')
exclude_columns:
- updated_at_ts
- inserted_at_ts
- Run
dbt docs generate
ordbt compile
and get the error:
Runtime Error
Compilation Error in test dbt_utils_equality_core_users_comparison_source_core_users___updated_at_ts__inserted_at_ts (models/core/core_users.yml)
`original`.`core`.`core_users` has less columns than `new`.`dbt_gdown_core`.`core_users`, please ensure they have the same columns or use the `compare_columns` or `exclude_columns` arguments to subset them.
> in macro default__test_equality (macros/generic_tests/equality.sql)
> called by macro test_equality (macros/generic_tests/equality.sql)
- If you build the model, then the error goes away:
dbt run --select core_users
Expected results
The project can both compile and build docs without models existing on disk.
Actual results
Compilation error regarding different number of columns in the models
Screenshots and log output
System information
The contents of your packages.yml
file:
packages:
- package: dbt-labs/dbt_utils
version: [">=1.0.0", "<1.3.0"]
- package: dbt-labs/codegen
version: [">=0.12.0"]
- package: calogica/dbt_date
version: [">=0.7.0", "<0.11.0"]
- package: dbt-labs/dbt_project_evaluator
version: [">=0.14.2", "<0.15.0"]
Which database are you using dbt with?
- postgres
- redshift
- bigquery
- snowflake
- other (specify: databricks)
The output of dbt --version
:
dbt Cloud CLI - 0.38.14 (9e5a178619e1048fafd2335581c36279db6a3429 2024-09-18T20:18:04Z)
However, using versionless
dbt within dbt Cloud.
Additional context
I believe it is related to this line of code. Here, it's just assuming both relations exist, but if the relation doesn't exist, then adapter.get_columns_in_relation
returns an empty list, so on line 65 in there, it sees the two lists are different and produces the error message. As an aside here, the error message is misleading, because it assumes that if the two models don't have the same columns, then the model that it is being compared to is the one with fewer columns, but that might not be the case - as here.
Activity