Error when using mocked databricks oauth profile #1274
Description
Describe the bug
The dbt databricks connector attempts to connect to databricks during dbt ls even though it doesn't need to, breaking generation of DAGs in Cosmos in CI environments.
Steps To Reproduce
Use a mocked profile like
derived-eng:
outputs:
cosmos_target:
auth_type: oauth
client_id: mock_value
client_secret: mock_value
host: mock_value
http_path: mock_value
schema: derived_eng
type: databricks
target: cosmos_targetto rub dbt ls on a project.
Expected behavior
dbt ls should return normally, not failing because the credentials are mocked.
Screenshots and log output
23:08:35 Running with dbt=1.10.15
WARNING:thrift.transport.sslcompat:using legacy validation callback
23:08:36 Databricks adapter: Failed to authenticate with legacy-azure-client-secret, trying oauth-m2m next. Error: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: host=https://mock_value, azure_client_secret=***, azure_client_id=mock_value, auth_type=azure-client-secret
23:13:37 Databricks adapter: Failed to authenticate with oauth-m2m. No more authentication methods to try. Error: default auth: oauth-m2m: Timed out after 0:05:00. Config: host=https://mock_value, client_id=mock_value, client_secret=***, auth_type=oauth-m2m
23:13:37 Encountered an error:
All authentication methods failed. Details: [('legacy-azure-client-secret', ValueError('default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: host=https://mock_value, azure_client_secret=***, azure_client_id=mock_value, auth_type=azure-client-secret')), ('oauth-m2m', ValueError('default auth: oauth-m2m: Timed out after 0:05:00. Config: host=https://mock_value, client_id=mock_value, client_secret=***, auth_type=oauth-m2m'))]
23:13:37 Traceback (most recent call last):
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/urllib3/util/connection.py", line 60, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/socket.py", line 978, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
raise new_e
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
self._validate_conn(conn)
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
conn.connect()
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/urllib3/connection.py", line 753, in connect
self.sock = sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/urllib3/connection.py", line 205, in _new_conn
raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0xf48bf9e9a5a0>: Failed to resolve 'mock_value' ([Errno -2] Name or service not known)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/requests/adapters.py", line 644, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/urllib3/util/retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='mock_value', port=443): Max retries exceeded with url: /oidc/.well-known/oauth-authorization-server (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0xf48bf9e9a5a0>: Failed to resolve 'mock_value' ([Errno -2] Name or service not known)"))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/retries.py", line 36, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/_base_client.py", line 283, in _perform
response = self._session.request(
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/requests/adapters.py", line 677, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='mock_value', port=443): Max retries exceeded with url: /oidc/.well-known/oauth-authorization-server (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0xf48bf9e9a5a0>: Failed to resolve 'mock_value' ([Errno -2] Name or service not known)"))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/credentials_provider.py", line 1053, in __call__
header_factory = provider(cfg)
^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/credentials_provider.py", line 75, in __call__
return self._headers_provider(cfg)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/credentials_provider.py", line 116, in wrapper
return func(cfg)
^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/credentials_provider.py", line 180, in oauth_service_principal
oidc = cfg.oidc_endpoints
^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/config.py", line 348, in oidc_endpoints
return get_workspace_endpoints(self.host)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/oauth.py", line 393, in get_workspace_endpoints
resp = client.do("GET", oidc)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/_base_client.py", line 196, in do
response = call(
^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/retries.py", line 65, in wrapper
raise TimeoutError(f"Timed out after {timeout}") from last_err
TimeoutError: Timed out after 0:05:00
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/config.py", line 533, in init_auth
self._header_factory = self._credentials_strategy(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/credentials_provider.py", line 1060, in __call__
raise ValueError(f"{auth_type}: {e}") from e
ValueError: oauth-m2m: Timed out after 0:05:00
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/config.py", line 197, in __init__
self.init_auth()
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/config.py", line 538, in init_auth
raise ValueError(f"{self._credentials_strategy.auth_type()} auth: {e}") from e
ValueError: default auth: oauth-m2m: Timed out after 0:05:00
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/adapters/databricks/credentials.py", line 361, in __post_init__
self._config = auth_methods[auth_type]()
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/adapters/databricks/credentials.py", line 302, in authenticate_with_oauth_m2m
return Config(
^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/databricks/sdk/config.py", line 201, in __init__
raise ValueError(message) from e
ValueError: default auth: oauth-m2m: Timed out after 0:05:00. Config: host=https://mock_value, client_id=mock_value, client_secret=***, auth_type=oauth-m2m
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/cli/requires.py", line 178, in wrapper
result, success = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/cli/requires.py", line 128, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/cli/requires.py", line 268, in wrapper
profile = load_profile(flags.PROJECT_DIR, flags.VARS, flags.PROFILE, flags.TARGET, threads)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/config/runtime.py", line 78, in load_profile
profile = Profile.render(
^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/config/profile.py", line 402, in render
return cls.from_raw_profiles(
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/config/profile.py", line 368, in from_raw_profiles
return cls.from_raw_profile_info(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/config/profile.py", line 324, in from_raw_profile_info
credentials: Credentials = cls._credentials_from_profile(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/config/profile.py", line 151, in _credentials_from_profile
credentials = cls.from_dict(data)
^^^^^^^^^^^^^^^^^^^
File "<string>", line 4, in __mashumaro_from_dict__
File "<string>", line 105, in __mashumaro_from_dict__
File "<string>", line 24, in __init__
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/adapters/databricks/credentials.py", line 133, in __post_init__
self._credentials_manager = DatabricksCredentialManager.create_from(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/adapters/databricks/credentials.py", line 283, in create_from
return DatabricksCredentialManager(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<string>", line 12, in __init__
File "/usr/local/airflow/dbt_venv/lib/python3.12/site-packages/dbt/adapters/databricks/credentials.py", line 381, in __post_init__
raise Exception(f"All authentication methods failed. Details: {exceptions}")
Exception: All authentication methods failed. Details: [('legacy-azure-client-secret', ValueError('default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: host=https://mock_value, azure_client_secret=***, azure_client_id=mock_value, auth_type=azure-client-secret')), ('oauth-m2m', ValueError('default auth: oauth-m2m: Timed out after 0:05:00. Config: host=https://mock_value, client_id=mock_value, client_secret=***, auth_type=oauth-m2m'))]
System information
The output of dbt --version:
Core:
- installed: 1.10.15
- latest: 1.10.15 - Up to date!
Plugins:
- spark: 1.9.3 - Up to date!
- databricks: 1.11.0 - Update available!
The operating system you're using:
Unix in docker
The output of python --version:
Python 3.12.9
Additional context
Came up while trying to fix astronomer/astronomer-cosmos#2164.