Description
Describe the bug
While upgrading to 0.6.4 from 0.6.3, we hit an issue that is similar to #350 but for a postgres target that had foreign data wrappers set up while using get_relations_by_pattern
Steps to reproduce
Setup an imported schema extdb_extschema
in postgres target database with table that matches '%_'+table_suffix
.
Reference in model to get_relations_by_pattern
:
{% set relations = dbt_utils.get_relations_by_pattern(schema_pattern='extdb__extschema',
table_pattern='%_'+table_suffix,
database=this.database) %}
Run dbt compile
on model for target database
Expected results
The successfully retrieves the set of relations to iterate over creating a single model out of the records in the matching tables (e.g. union all
of matching relations).
This is the current result when running dbt utils 0.6.3
Actual results
Unhandled runtime errors occur during the get_relations_by_pattern
call causing the model to not build.
Screenshots and log output
Unhandled error while executing model.test_model.consolidation
Field "type" of type typing.Union[dbt.contracts.relation.RelationType, NoneType] in dbt.adapters.postgres.relation.PostgresRelation has invalid value 'foreign'
System information
The contents of your packages.yml
file:
packages:
- package: fishtown-analytics/dbt_utils
version: 0.6.4
dbt version: 0.19.1
Which database are you using dbt with?
- postgres
- redshift
- bigquery
- snowflake
- other (specify: ____________)
The output of dbt --version
:
installed version: 0.19.1
latest version: 0.19.1
Up to date!
Plugins:
- snowflake: 0.19.1
- redshift: 0.19.1
- bigquery: 0.19.1
- postgres: 0.19.1
Additional context
The addition of table_type
into the default__get_tables_by_pattern_sql
macro causes problems with foreign tables in postgres since they have a type of FOREIGN
in information_schema
, but dbt
doesn't support that type by that name.
Are you interested in contributing the fix?
Yeah. I'm not sure if there are any caveats about what the external
type means in dbt
core or weird downstream effects of it, but a similar fix to #351 should solve the problem unless there are concerns about ramifications in other adapters.
Activity