Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(azure_ad): make redirect and graph_url optional parameters and update docs #4754

Merged
merged 1 commit into from
Apr 27, 2022
Merged

fix(azure_ad): make redirect and graph_url optional parameters and update docs #4754

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions metadata-ingestion/source_docs/azure-ad.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@ to read your organization's Users and Groups. The following permissions are requ
- `GroupMember.Read.All`
- `User.Read.All`

You can add a permission by navigating to the permissions tab in your DataHub application on the Azure AD portal. ![Azure AD API Permissions](./images/azure_ad_api_permissions.png)

You can view the necessary endpoints to configure by clicking on the Endpoints button in the Overview tab. ![Azure AD Endpoints](./images/azure_ad_endpoints.png)

You can use the following recipe to get started with Azure ingestion! See [below](#config-details) for full configuration options.

```yml
Expand All @@ -88,9 +92,10 @@ source:
client_id: "00000000-0000-0000-0000-000000000000"
tenant_id: "00000000-0000-0000-0000-000000000000"
client_secret: "xxxxx"
redirect: "https://login.microsoftonline.com/common/oauth2/nativeclient"
authority: "https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000"
token_url: "https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/token"
# All of the below parameters are optional.
redirect: "https://login.microsoftonline.com/common/oauth2/nativeclient"
graph_url: "https://graph.microsoft.com/v1.0"
ingest_users: True
ingest_groups: True
Expand All @@ -116,11 +121,11 @@ Note that a `.` is used to denote nested fields in the YAML configuration block.
|----------------------------------------|-----------------|----------|-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `client_id` | string | ✅ | | Application ID. Found in your app registration on Azure AD Portal |
| `tenant_id` | string | ✅ | | Directory ID. Found in your app registration on Azure AD Portal |
| `client_secret` | string | ✅ | | Client secret. Found in your app registration on Azure AD Portal |
| `redirect` | string | ✅ | | Redirect URI. Found in your app registration on Azure AD Portal |
| `client_secret` | string | ✅ | | Client secret. Found in your app registration on Azure AD Portal |
| `authority` | string | ✅ | | The [authority](https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-client-application-configuration) is a URL that indicates a directory that MSAL can request tokens from. |
| `token_url` | string | ✅ | | The token URL that acquires a token from Azure AD for authorizing requests. This source will only work with v1.0 endpoint. |
| `graph_url` | string | ✅ | | [Microsoft Graph API endpoint](https://docs.microsoft.com/en-us/graph/use-the-api) |
| `redirect` | string | | | Redirect URI. Found in your app registration on Azure AD Portal. Defaults to https://login.microsoftonline.com/common/oauth2/nativeclient. |
| `graph_url` | string | | | [Microsoft Graph API endpoint](https://docs.microsoft.com/en-us/graph/use-the-api). Defaults to https://graph.microsoft.com/v1.0. |
| `ingest_users` | bool | | `True` | Whether users should be ingested into DataHub. |
| `ingest_groups` | bool | | `True` | Whether groups should be ingested into DataHub. |
| `ingest_group_membership` | bool | | `True` | Whether group membership should be ingested into DataHub. ingest_groups must be True if this is True. |
Expand Down
Binary file added metadata-ingestion/source_docs/images/azure_ad_api_permissions.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added metadata-ingestion/source_docs/images/azure_ad_endpoints.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 7 additions & 3 deletions metadata-ingestion/src/datahub/ingestion/source/identity/azure_ad.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,12 @@ class AzureADConfig(ConfigModel):
client_id: str
tenant_id: str
client_secret: str
redirect: str
authority: str
token_url: str
graph_url: str

# Optional: URLs for redirect and hitting the Graph API
redirect: str = "https://login.microsoftonline.com/common/oauth2/nativeclient"
graph_url: str = "https://graph.microsoft.com/v1.0"

# Optional: Customize the mapping to DataHub Username from an attribute in the REST API response
# Reference: https://docs.microsoft.com/en-us/graph/api/user-list?view=graph-rest-1.0&tabs=http#response-1
Expand Down Expand Up @@ -228,7 +230,9 @@ def _add_group_members_to_group_membership(
)
else:
# Unless told otherwise, we only care about users and groups. Silently skip other object types.
pass
logger.warning(
aditya-radhakrishnan marked this conversation as resolved.
Show resolved Hide resolved
f"Unsupported @odata.type '{odata_type}' found in Azure group member. Skipping...."
)

def _add_user_to_group_membership(
self,
Expand Down