1.0.0-rc1
Pre-releaseThis is the first release candidate for dbt utils 1.0. A full migration guide will accompany the final release, but here is the changelog:
New
- New macro:
get_single_value()
- New macro:
safe_divide()
- New test:
not_empty_string
Enhancements
group_by_columns
in some tests- Able to prevent
quote_identifiers
instar()
Fixes
union
now includes/excludes columns case-insensitivelyslugify
prefixes an underscore when the first char is a digitexpression_is_true
doesn’t output*
unless storing failuresstar()
only returns a*
when no columns are found if running under thedbt compile
command orexecute=false
mode. Returning*
in these contexts allows SQLFluff's linting to work, but returning nothing under standard conditions ensure that you don't accidentally pull all your columns if you meant to exclude them all.recency
test can now be configured to ignore the time component, useful when your tested column is a date instead of a datetime.
Deprecated
- The
expression_is_true
test no longer has a dedicatedcondition
argument. Instead, usewhere
which is now available natively to all tests - The
unique_where
andnot_null_where
tests have been removed, because where is now available natively to all tests. - 🚨
surrogate_key()
has been replaced bygenerate_surrogate_key()
. The original treated null values and blank strings the same, which could lead to duplicate keys being created.generate_surrogate_key()
does not have this flaw. Compare the surrogate keys calculated for these columns
Changing the calculation method for surrogate keys, even for the better, could have significant consequences in downstream uses (such as snapshots and incremental models which use this column as their unique_key
). Because of this, it is possible to opt in to the legacy behaviour by setting the following variable in your dbt project:
#dbt_project.yml
vars:
surrogate_key_treat_nulls_as_empty_strings: true #turn on legacy behaviour
By creating a new macro instead of updating the behaviour of the old one, we are requiring all projects who use this macro to make an explicit decision about which approach is better for their context. Our recommendation is that existing users should opt into the legacy behaviour unless you are confident that either a) your surrogate keys never contained nulls, or b) your surrogate keys are not used for incremental models, snapshots or other stateful artifacts and so can be regenerated with new values without issue.
Warning to package maintainers: you can not assume one behaviour or the other, as it can be enabled/disabled by the end user.
- Deprecation:
current_timestamp()
replaced bydbt.current_timestamp()
. Note that Postgres and Snowflake’s implementation ofcurrent_timestamp()
differs between the olddbt_utils
one and the newdbt
one (full details here). For those adapters, usedbt.current_timestamp_backcompat()
if you need identical, backwards-compatible behaviour. This discrepancy will hopefully be reconciled in some future version of dbt Core (maybe 2.0?) - All other cross-db macros have moved to dbt namespace with no changes necessary other than swapping out
dbt_utils.
fordbt.
; see https://docs.getdbt.com/reference/dbt-jinja-functions/cross-database-macros insert_by_period
materialization moved to experimental reposafe_add()
only works with a list of argumentsdeduplicate()
changes:group_by
argument is removed, replaced bypartition_by
relation_alias
is removed, replaced by passing the alias directly to therelation
argumentorder_by
is now mandatory. Pass a static value like1
if you don’t care how they are deduplicated
table
argument has been removed fromunpivot
. Userelation
instead
Full Changelog: 0.9.6...1.0.0-rc1