Authentication failure occurs in Azure Databricks when using dbt-databricks 1.6
#609
Labels
bug
Something isn't working
dbt-databricks 1.6
#609
Describe the bug
Authorization of app registration fails at some point. Tested when running
dbt --debug debug
, on Azure Databricks.Steps To Reproduce
dbt --debug debug
dbt-databricks==1.6.8
Expected behavior
Expect job to run successfully and show that connection was possible.
Screenshots and log output
Output of job:
System information
The output of
dbt --version
:`1.6.10`` (see above output)
Output logs from
dbt debug
As above
The operating system you're using:
DBR 13.3 LTS
Apache Spark 3.4.1
The output of
python --version
:Python 3.10.12
Additional context
databricks-sdk
released a new version 0.18.0dbt-core==1.6.x
dbt-databricks==1.7.8
databricks-sdk
The
setup.py
specifies as of v1.6.8 that we can install anythingdatabricks-sdk>= 0.9.0
(despite requirements.txt specifyingdatabricks-sdk==0.9.0
). Thus my proposed workarounds:dbt
is: fix thedatabricks-sdk
versiondatabricks-sdk==0.17.0
. Adding this lib to the cluster keeps the dependency resolves within the range and auth is successful.Note: I have not tested for other side-effects of fixing the
databricks-sdk
version... I had other issues come up, not sure if they were related to this fix so I gave up and upgraded to dbt 1.7The text was updated successfully, but these errors were encountered: