feat(ingest/transformer): tags to terms transformer #10758
Conversation
github-actions
bot
added
ingestion
sagar-salvi-apptware
force-pushed
the
fix/ING-647/tags-to-term-transformer
branch
2 times, most recently
from
66a187c to
3b0eeef
Compare
sagar-salvi-apptware
force-pushed
the
fix/ING-647/tags-to-term-transformer
branch
from
3b0eeef to
c417c01
Compare
sagar-salvi-apptware
force-pushed
the
fix/ING-647/tags-to-term-transformer
branch
2 times, most recently
from
4165b16 to
3d1dfc3
Compare
sagar-salvi-apptware
force-pushed
the
fix/ING-647/tags-to-term-transformer
branch
from
3d1dfc3 to
050c475
Compare
WalkthroughThe recent changes introduce a new transformer, Changes
Poem
Tip Early access features: enabledWe are currently testing the following features in early access:
Note:
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
anshbansal
changed the title
anshbansal
added
the
merge-pending-ci
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Outside diff range and nitpick comments (47)
metadata-ingestion/docs/transformer/dataset_transformer.md (46)
Line range hint
24-24: Fix grammatical errors.Correct the sentence to: "Whether it should be considered a user or not."
- | `is_user` | | bool | `true` | Whether should be consider a user or not. If `false` then considered a group. | + | `is_user` | | bool | `true` | Whether it should be considered a user or not. If `false`, then considered a group. |Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
46-46: Add a comma after 'Hence'.A comma is needed after the conjunctive adverb 'Hence'.
- Hence users `[email protected]` and `[email protected]` will be added as owners. + Hence, users `[email protected]` and `[email protected]` will be added as owners.Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
68-68: Fix subject-verb agreement.Change 'characters needs' to 'characters need'.
- Add owners, however some tag characters needs to replace with some other characters before extracting owner. + Add owners, however some tag characters need to be replaced with some other characters before extracting owner.Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
80-80: Fix grammatical error.Change 'owner type also need to extracted' to 'owner type also needs to be extracted'.
- Add owners, however owner type also need to extracted from tag pattern. + Add owners, however owner type also needs to be extracted from tag pattern.Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
96-96: Correct article usage.Change 'a Onwer URN' to 'an Owner URN'.
- Matches against a Onwer URN and remove the matching part from the Owner URN. + Matches against an Owner URN and remove the matching part from the Owner URN.Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
128-128: Fix subject-verb agreement.Change 'ownership type of the owners' to 'ownership type of the owners'.
- ownership type of the owners (either as enum or ownership type urn) + ownership type of the owners (either as enum or ownership type urns)Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
132-132: Add missing preposition.Add 'to' before 'section'.
- please refer section [Relationship Between replace_existing And semantics] + please refer to section [Relationship Between replace_existing And semantics]Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
153-153: Complete the sentence.Add a subject to form a complete sentence.
- `simple_add_dataset_ownership` can be configured in below different way + The `simple_add_dataset_ownership` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
179-179: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add owners, however keep the owners available for the dataset on DataHub GMS + Add owners, however, keep the owners available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
197-197: Fix subject-verb agreement.Change 'ownership type of the owners' to 'ownership type of the owners'.
- ownership type of the owners (either as enum or ownership type urn) + ownership type of the owners (either as enum or ownership type urns)Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
216-216: Complete the sentence.Add a subject to form a complete sentence.
- `pattern_add_dataset_ownership` can be configured in below different way + The `pattern_add_dataset_ownership` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
242-242: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add owner, however keep the owners available for the dataset on DataHub GMS + Add owner, however, keep the owners available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
271-271: Add a comma after 'Currently'.A comma is needed after the conjunctive adverb 'Currently'.
- Currently only `urn` is supported. + Currently, only `urn` is supported.Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
316-316: Complete the sentence.Add a subject to form a complete sentence.
- `simple_add_dataset_tags` can be configured in below different way + The `simple_add_dataset_tags` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
370-370: Complete the sentence.Add a subject to form a complete sentence.
- `pattern_add_dataset_tags` can be configured in below different way + The `pattern_add_dataset_tags` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
372-372: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add tags, however replace existing tags sent by ingestion source + Add tags, however, replace existing tags sent by ingestion sourceTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
449-449: Complete the sentence.Add a subject to form a complete sentence.
- `add_dataset_tags` can be configured in below different way + The `add_dataset_tags` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
451-451: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add tags, however replace existing tags sent by ingestion source + Add tags, however, replace existing tags sent by ingestion sourceTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
497-497: Add a comma after 'name'.A comma is needed after the phrase 'the database instance name'.
- browse path like the database instance name you can use this. + browse path like the database instance name, you can use this.Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
530-530: Complete the sentence.Add a subject to form a complete sentence.
- `set_dataset_browse_path` can be configured in below different way + The `set_dataset_browse_path` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
582-582: Complete the sentence.Add a subject to form a complete sentence.
- `simple_add_dataset_terms` can be configured in below different way + The `simple_add_dataset_terms` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
594-594: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add terms, however overwrite the terms available for the dataset on DataHub GMS + Add terms, however, overwrite the terms available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
604-604: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add terms, however keep the terms available for the dataset on DataHub GMS + Add terms, however, keep the terms available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
635-635: Complete the sentence.Add a subject to form a complete sentence.
- `pattern_add_dataset_terms` can be configured in below different way + The `pattern_add_dataset_terms` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
649-649: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add terms, however overwrite the terms available for the dataset on DataHub GMS + Add terms, however, overwrite the terms available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
660-660: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add terms, however keep the terms available for the dataset on DataHub GMS + Add terms, however, keep the terms available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
743-743: Complete the sentence.Add a subject to form a complete sentence.
- `pattern_add_dataset_schema_terms` can be configured in below different way + The `pattern_add_dataset_schema_terms` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
756-756: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add terms, however overwrite the terms available for the dataset on DataHub GMS + Add terms, however, overwrite the terms available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
767-767: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add terms, however keep the terms available for the dataset on DataHub GMS + Add terms, however, keep the terms available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
803-803: Complete the sentence.Add a subject to form a complete sentence.
- `pattern_add_dataset_schema_tags` can be configured in below different way + The `pattern_add_dataset_schema_tags` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
805-805: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add tags, however replace existing tag sent by ingestion source + Add tags, however, replace existing tag sent by ingestion sourceTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
846-846: Add missing article.Add 'the' before 'dataset entity'.
- `simple_add_dataset_properties` transformer assigns the properties to dataset entity from the configuration. + `simple_add_dataset_properties` transformer assigns the properties to the dataset entity from the configuration.Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
847-847: Add a comma after 'Note'.A comma is needed after the phrase 'Note'.
- Note in case of any key collision, the value in the config will overwrite the previous value. + Note, in case of any key collision, the value in the config will overwrite the previous value.Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
859-859: Complete the sentence.Add a subject to form a complete sentence.
- `simple_add_dataset_properties` can be configured in below different way + The `simple_add_dataset_properties` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
926-926: Complete the sentence.Add a subject to form a complete sentence.
- `add_dataset_properties` can be configured in below different way + The `add_dataset_properties` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1018-1018: Complete the sentence.Add a subject to form a complete sentence.
- `simple_add_dataset_domain` can be configured in below different way + The `simple_add_dataset_domain` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1020-1020: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add domains, however replace existing domains sent by ingestion source + Add domains, however, replace existing domains sent by ingestion sourceTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1030-1030: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add domains, however overwrite the domains available for the dataset on DataHub GMS + Add domains, however, overwrite the domains available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1040-1040: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add domains, however keep the domains available for the dataset on DataHub GMS + Add domains, however, keep the domains available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1055-1055: Fix grammatical error.Change 'need to be apply' to 'need to be applied'.
- simple domain name or domain urn need to be apply on matching dataset urn. + simple domain name or domain urn need to be applied on matching dataset urn.Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1077-1077: Complete the sentence.Add a subject to form a complete sentence.
- `pattern_add_dataset_domain` can be configured in below different way + The `pattern_add_dataset_domain` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1079-1079: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add domains, however replace existing domains sent by ingestion source + Add domains, however, replace existing domains sent by ingestion sourceTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1090-1090: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add domains, however overwrite the domains available for the dataset on DataHub GMS + Add domains, however, overwrite the domains available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1101-1101: Add a comma after 'however'.A comma is needed after the adverb 'however'.
- Add domains, however keep the domains available for the dataset on DataHub GMS + Add domains, however, keep the domains available for the dataset on DataHub GMSTools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1133-1133: Fix repeated word.Remove the repeated word 'tag'.
- For example, instead of using the tag URN urn:li:tag:NeedsDocumentation, + For example, instead of using the tag URN urn:li:NeedsDocumentation,Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
Line range hint
1144-1144: Complete the sentence.Add a subject to form a complete sentence.
- `domain_mapping_based_on_tags` can be configured in below different way + The `domain_mapping_based_on_tags` transformer can be configured in the following ways:Tools
Markdownlint
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
metadata-ingestion/tests/unit/test_transform_dataset.py (1)
Line range hint
126-126: Avoid function calls in default arguments.Do not perform function call
models.StatusClassin argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable.- aspects: Optional[List[Any]] = [models.StatusClass(removed=False)], + aspects: Optional[List[Any]] = None, ) -> models.MetadataChangeEventClass: if aspects is None: aspects = [models.StatusClass(removed=False)]
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (6)
- metadata-ingestion/docs/transformer/dataset_transformer.md (2 hunks)
- metadata-ingestion/setup.py (1 hunks)
- metadata-ingestion/src/datahub/ingestion/graph/client.py (1 hunks)
- metadata-ingestion/src/datahub/ingestion/transformer/dataset_transformer.py (2 hunks)
- metadata-ingestion/src/datahub/ingestion/transformer/tags_to_terms.py (1 hunks)
- metadata-ingestion/tests/unit/test_transform_dataset.py (4 hunks)
Additional context used
Ruff
metadata-ingestion/src/datahub/ingestion/graph/client.py
317-317: Yoda conditions are discouraged, use
aspect == aspect_type.ASPECT_NAMEinsteadReplace Yoda condition with
aspect == aspect_type.ASPECT_NAME(SIM300)
724-724: Use
enumerate()for index variableentities_yieldedinforloop(SIM113)
1180-1180: Use of
functools.lru_cacheorfunctools.cacheon methods can lead to memory leaks(B019)
1282-1292: Use f-string instead of
formatcallConvert to f-string
(UP032)
1333-1333: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1352-1352: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1356-1365: Use format specifiers instead of percent format
Replace with format specifiers
(UP031)
1385-1385: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1389-1405: Use format specifiers instead of percent format
Replace with format specifiers
(UP031)
1424-1424: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1425-1425: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1429-1445: Use format specifiers instead of percent format
Replace with format specifiers
(UP031)
1465-1465: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
metadata-ingestion/tests/unit/test_transform_dataset.py
126-126: Do not perform function call
models.StatusClassin argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable(B008)
1901-1901: Do not perform function call
PipelineContextin argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable(B008)
LanguageTool
metadata-ingestion/docs/transformer/dataset_transformer.md
[uncategorized] ~24-~24: Between ‘whether’ and ‘should’ a pronoun, proper noun or plural noun seems to be missing.
Context: ... | | bool |true| Whether should be consider a user or not. Iffalset...(LACK_OF_PRP_OR_NOUN_FROM_WHETHER_SHOULD)
[grammar] ~24-~24: There may an error in the verb form ‘be consider’.
Context: ... | bool |true| Whether should be consider a user or not. Iffalsethen consider...(MD_BE_NON_VBP)
[uncategorized] ~46-~46: A comma may be missing after the conjunctive/linking adverb ‘Hence’.
Context: ...attern will be converted into an owner. Hence users[email protected]and `[email protected]...(SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA)
[uncategorized] ~68-~68: Possible missing comma found.
Context: ...ype:data_product" ``` - Add owners, however some tag characters needs to replace wi...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~68-~68: The verb form ‘needs’ does not seem to match the subject ‘characters’.
Context: ...Add owners, however some tag characters needs to replace with some other characters b...(SUBJECT_VERB_AGREEMENT_PLURAL)
[uncategorized] ~80-~80: Possible missing comma found.
Context: ... "__": "_" ``` - Add owners, however owner type also need to extracted from ...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~80-~80: It appears that the verb form is incorrect or a word is missing.
Context: ...owners, however owner type also need to extracted from tag pattern. For example: from tag...(HAS_TO_APPROVED_BY)
[misspelling] ~96-~96: Use “an” instead of ‘a’ if the following word starts with a vowel sound, e.g. ‘an article’, ‘an hour’.
Context: ...om the Owner URN(s) | Matches against a Onwer URN and remove the matching part ...(EN_A_VS_AN)
[grammar] ~128-~128: In this context, ‘type’ should agree in number with the noun after ‘of’.
Context: ... string | "DATAOWNER" | ownership type of the owners (either as enum or ownership type urn) ...(TYPE_OF_PLURAL)
[uncategorized] ~132-~132: Possible missing preposition found.
Context: ...existingandsemantics`, please refer section [Relationship Between replace_existing ...(AI_HYDRA_LEO_MISSING_TO)
[style] ~153-~153: To form a complete sentence, be sure to include a subject.
Context: ...R" ```simple_add_dataset_ownershipcan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~179-~179: Possible missing comma found.
Context: ...ystem__producer" ``` - Add owners, however keep the owners available for the datas...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~197-~197: In this context, ‘type’ should agree in number with the noun after ‘of’.
Context: ... | "DATAOWNER" | ownership type of the owners (either as enum or ownership type urn) ...(TYPE_OF_PLURAL)
[style] ~216-~216: To form a complete sentence, be sure to include a subject.
Context: ... ```pattern_add_dataset_ownershipcan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~242-~242: Possible missing comma found.
Context: ...system__producer" ``` - Add owner, however keep the owners available for the datas...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~271-~271: A comma may be missing after the conjunctive/linking adverb ‘Currently’.
Context: ... | Which field to extract tag from. Currently onlyurnis supported. | | `extract_...(SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA)
[style] ~316-~316: To form a complete sentence, be sure to include a subject.
Context: ...egacy" ```simple_add_dataset_tagscan be configured in below different way ...(MISSING_IT_THERE)
[style] ~370-~370: To form a complete sentence, be sure to include a subject.
Context: ...ion"] ```pattern_add_dataset_tagscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~372-~372: Possible missing comma found.
Context: ...ed in below different way - Add tags, however replace existing tags sent by ingestion...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~449-~449: To form a complete sentence, be sure to include a subject.
Context: ...lling-the-package).add_dataset_tagscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~451-~451: Possible missing comma found.
Context: ...ed in below different way - Add tags, however replace existing tags sent by ingestion...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~497-~497: Possible missing comma found.
Context: ... browse path like the database instance name you can use this. ```yaml transformers:...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~530-~530: To form a complete sentence, be sure to include a subject.
Context: ...e transform.set_dataset_browse_pathcan be configured in below different way ...(MISSING_IT_THERE)
[style] ~582-~582: To form a complete sentence, be sure to include a subject.
Context: ...ress" ```simple_add_dataset_termscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~594-~594: Possible missing comma found.
Context: ...ssaryTerm:Address" ``` - Add terms, however overwrite the terms available for the d...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~604-~604: Possible missing comma found.
Context: ...ssaryTerm:Address" ``` - Add terms, however keep the terms available for the datase...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~635-~635: To form a complete sentence, be sure to include a subject.
Context: ...de"] ```pattern_add_dataset_termscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~649-~649: Possible missing comma found.
Context: ...Term:PostalCode"] ``` - Add terms, however overwrite the terms available for the d...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~660-~660: Possible missing comma found.
Context: ...yTerm:PostalCode"] ``` - Add terms, however keep the terms available for the datase...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~697-~697: To form a complete sentence, be sure to include a subject.
Context: ... - "tag_name" ```tags_to_termcan be configured in below different way -...(MISSING_IT_THERE)
[style] ~743-~743: To form a complete sentence, be sure to include a subject.
Context: ...```pattern_add_dataset_schema_termscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~756-~756: Possible missing comma found.
Context: ...lossaryTerm:Name"] ``` - Add terms, however overwrite the terms available for the d...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~767-~767: Possible missing comma found.
Context: ...lossaryTerm:Name"] ``` - Add terms, however keep the terms available for the datase...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~803-~803: To form a complete sentence, be sure to include a subject.
Context: ... ```pattern_add_dataset_schema_tagscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~805-~805: Possible missing comma found.
Context: ...ed in below different way - Add tags, however replace existing tag sent by ingestion ...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~846-~846: Possible missing article found.
Context: ...transformer assigns the properties to dataset entity from the configuration.propert...(AI_HYDRA_LEO_MISSING_THE)
[uncategorized] ~847-~847: Possible missing comma found.
Context: ...field is a dictionary of string values. Note in case of any key collision, the value...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~859-~859: To form a complete sentence, be sure to include a subject.
Context: ... ```simple_add_dataset_propertiescan be configured in below different way ...(MISSING_IT_THERE)
[style] ~926-~926: To form a complete sentence, be sure to include a subject.
Context: ...perties ```add_dataset_propertiescan be configured in below different way ...(MISSING_IT_THERE)
[style] ~1018-~1018: To form a complete sentence, be sure to include a subject.
Context: ...ering ```simple_add_dataset_domaincan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~1020-~1020: Possible missing comma found.
Context: ...in below different way - Add domains, however replace existing domains sent by ingest...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~1030-~1030: Possible missing comma found.
Context: ...urn:li:domain:hr" ``` - Add domains, however overwrite the domains available for the...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~1040-~1040: Possible missing comma found.
Context: ...rn:li:domain:hr" ``` - Add domains, however keep the domains available for the data...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~1055-~1055: The past participle is required after “to be”.
Context: ...le domain name or domain urn need to be apply on matching dataset urn. | | `replace_e...(BE_VBP_IN)
[style] ~1077-~1077: To form a complete sentence, be sure to include a subject.
Context: ...e"] ```pattern_add_dataset_domaincan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~1079-~1079: Possible missing comma found.
Context: ...in below different way - Add domains, however replace existing domains sent by ingest...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~1090-~1090: Possible missing comma found.
Context: ...omain:finance"] ``` - Add domains, however overwrite the domains available for the...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~1101-~1101: Possible missing comma found.
Context: ...omain:finance"] ``` - Add domains, however keep the domains available for the data...(AI_HYDRA_LEO_MISSING_COMMA)
[duplication] ~1133-~1133: Possible typo: you repeated a word
Context: ... For example, instead of using the tag URN urn:li:tag:NeedsDocumentation, you should s...(ENGLISH_WORD_REPEAT_RULE)
[style] ~1144-~1144: To form a complete sentence, be sure to include a subject.
Context: ...n" ```domain_mapping_based_on_tagscan be configured in below different way -...(MISSING_IT_THERE)
[typographical] ~1237-~1237: The word “however” is an adverb that can’t be used like a conjunction, and therefore needs to be separated from the sentence.
Context: ...ontext ofsimple_add_dataset_ownership, however it is applicable for all dataset transf...(HOWEVER_SENTENCE)
[uncategorized] ~1364-~1364: Possible missing comma found.
Context: ..._dict) return cls(config, ctx) ``` Next we need to tell the helper classes whic...(AI_HYDRA_LEO_MISSING_COMMA)
[typographical] ~1374-~1374: Consider adding a comma after ‘Finally’ for more clarity.
Context: ...lf) -> str: return "ownership" ``` Finally we need to implement the `transform_asp...(RB_LY_COMMA)
Markdownlint
metadata-ingestion/docs/transformer/dataset_transformer.md
4-4: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
7-7: Expected: 0 or 2; Actual: 95
Trailing spaces(MD009, no-trailing-spaces)
113-113: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
123-123: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
153-153: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
192-192: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
216-216: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
288-288: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
316-316: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
370-370: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
449-449: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
518-518: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
530-530: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
568-568: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
582-582: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
635-635: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
743-743: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
803-803: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
859-859: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
926-926: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
991-991: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1018-1018: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1051-1051: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1059-1059: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1063-1063: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1077-1077: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1192-1192: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1248-1248: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1255-1255: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1261-1261: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1268-1268: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1302-1302: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1402-1402: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1404-1404: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1426-1426: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1427-1427: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1417-1417: Column: 1
Hard tabs(MD010, no-hard-tabs)
1517-1517: Column: 1
Hard tabs(MD010, no-hard-tabs)
95-95: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
152-152: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
294-294: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
786-786: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
962-962: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
979-979: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
990-990: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1017-1017: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1113-1113: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1114-1114: Expected: 1; Actual: 3
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1143-1143: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1270-1270: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1271-1271: Expected: 1; Actual: 3
Multiple consecutive blank lines(MD012, no-multiple-blanks)
4-4: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
19-19: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
20-20: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
20-20: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
89-89: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
90-90: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
90-90: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
107-107: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
108-108: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
108-108: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
123-123: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
123-123: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
124-124: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
124-124: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
192-192: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
193-193: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
193-193: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
255-255: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
267-267: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
267-267: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
268-268: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
268-268: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
295-295: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
296-296: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
296-296: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
348-348: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
348-348: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
349-349: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
349-349: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
405-405: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
405-405: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
406-406: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
406-406: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
475-475: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
475-475: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
476-476: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
476-476: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
560-560: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
561-561: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
561-561: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
615-615: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
616-616: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
616-616: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
672-672: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
673-673: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
721-721: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
722-722: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
722-722: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
778-778: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
778-778: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
779-779: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
779-779: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
838-838: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
838-838: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
839-839: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
839-839: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
892-892: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
893-893: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
893-893: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
955-955: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
956-956: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
956-956: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
973-973: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
974-974: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
974-974: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
991-991: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
992-992: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
992-992: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1051-1051: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1052-1052: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1052-1052: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1115-1115: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1116-1116: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1167-1167: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1168-1168: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1168-1168: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1186-1186: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1187-1187: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1187-1187: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1206-1206: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1207-1207: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1207-1207: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1236-1236: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
4-4: null
Multiple top-level headings in the same document(MD025, single-title, single-h1)
51-51: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
58-58: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
60-60: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
67-67: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
69-69: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
79-79: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
81-81: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
122-122: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
156-156: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
166-166: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
168-168: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
178-178: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
180-180: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
219-219: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
229-229: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
231-231: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
241-241: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
243-243: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
319-319: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
327-327: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
329-329: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
337-337: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
339-339: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
347-347: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
373-373: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
382-382: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
384-384: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
393-393: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
395-395: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
404-404: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
446-446: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
452-452: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
458-458: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
460-460: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
466-466: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
468-468: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
474-474: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
489-489: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
498-498: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
504-504: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
508-508: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
515-515: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
520-520: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
527-527: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
533-533: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
540-540: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
542-542: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
549-549: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
551-551: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
585-585: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
593-593: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
595-595: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
603-603: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
605-605: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
638-638: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
648-648: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
650-650: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
659-659: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
661-661: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
700-700: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
709-709: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
711-711: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
746-746: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
755-755: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
757-757: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
766-766: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
768-768: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
777-777: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
806-806: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
815-815: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
817-817: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
826-826: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
828-828: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
837-837: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
862-862: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
870-870: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
872-872: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
880-880: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
882-882: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
929-929: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
936-936: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
938-938: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
945-945: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
947-947: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
981-981: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1008-1008: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1021-1021: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1029-1029: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1031-1031: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1039-1039: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1041-1041: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1080-1080: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1089-1089: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1091-1091: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1100-1100: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1102-1102: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1147-1147: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1155-1155: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1157-1157: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1233-1233: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
43-43: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
289-289: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
347-347: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
404-404: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
474-474: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
484-484: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
699-699: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
710-710: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
710-710: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
777-777: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
837-837: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1245-1245: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1251-1251: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1258-1258: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1264-1264: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
17-17: null
Spaces inside link text(MD039, no-space-in-links)
17-17: Expected: leading_and_trailing; Actual: leading_only; Missing trailing pipe
Table pipe style(MD055, table-pipe-style)
Additional comments not posted (16)
metadata-ingestion/src/datahub/ingestion/transformer/dataset_transformer.py (4)
30-31: Class DocumentationThe class-level docstring is clear and concise.
33-35: Constructor InitializationThe constructor correctly calls the superclass initializer.
36-37: Entity Types MethodThe
entity_typesmethod correctly returns the list of entity types.
143-145: Aspect Name MethodThe
aspect_namemethod correctly returns the aspect name.metadata-ingestion/src/datahub/ingestion/transformer/tags_to_terms.py (7)
21-23: Class DocumentationThe class-level docstring is missing. Consider adding a docstring to describe the purpose and usage of this class.
class TagsToTermMapperConfig(TransformerSemanticsConfigModel): """ Configuration model for TagsToTermMapper. """ tags: List[str]
28-32: Constructor InitializationThe constructor correctly initializes the
ctxandconfigattributes.
33-37: Create MethodThe
createmethod correctly parses the configuration dictionary and returns an instance ofTagsToTermMapper.
39-59: Merge with Server Glossary TermsThe
_merge_with_server_glossary_termsmethod correctly handles the merging of terms.
62-67: Get Tags from Global TagsThe
get_tags_from_global_tagsmethod correctly extracts tags fromGlobalTagsClass.
70-82: Get Tags from Schema MetadataThe
get_tags_from_schema_metadatamethod correctly extracts tags fromSchemaMetadataClass.
84-145: Transform AspectThe
transform_aspectmethod correctly handles the transformation of aspects by mapping tags to glossary terms and updating the glossary terms.metadata-ingestion/setup.py (1)
718-718: Entry Point AdditionThe new transformer
tags_to_termhas been correctly added to the entry points.metadata-ingestion/tests/unit/test_transform_dataset.py (4)
3761-3796: LGTM!The function
test_tags_to_terms_with_no_matching_termsis well-structured and verifies the expected behavior correctly.
3798-3826: LGTM!The function
test_tags_to_terms_with_missing_tagsis well-structured and verifies the expected behavior correctly.
3829-3867: LGTM!The function
test_tags_to_terms_with_partial_matchis well-structured and verifies the expected behavior correctly.
Line range hint
135-140: LGTM!The function
make_generic_dataset_mcpis well-structured and creates the MCP correctly.
There was a problem hiding this comment.
Actionable comments posted: 6
Outside diff range and nitpick comments (1)
metadata-ingestion/tests/unit/test_transform_dataset.py (1)
Line range hint
126-126: Avoid using function calls in default argument values.Do not perform function call
models.StatusClassin argument defaults; instead, perform the call within the function.def make_generic_dataset( entity_urn: str = "urn:li:dataset:(urn:li:dataPlatform:bigquery,example1,PROD)", aspects: Optional[List[Any]] = None, ) -> models.MetadataChangeEventClass: if aspects is None: # Default to a status aspect if none is provided. - aspects = [models.StatusClass(removed=False)] + aspects = [default_status_aspect] return models.MetadataChangeEventClass( proposedSnapshot=models.DatasetSnapshotClass( urn=entity_urn, aspects=aspects, ), ) default_status_aspect = models.StatusClass(removed=False)
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (3)
- metadata-ingestion/docs/transformer/dataset_transformer.md (2 hunks)
- metadata-ingestion/src/datahub/ingestion/graph/client.py (1 hunks)
- metadata-ingestion/tests/unit/test_transform_dataset.py (4 hunks)
Additional context used
Ruff
metadata-ingestion/src/datahub/ingestion/graph/client.py
317-317: Yoda conditions are discouraged, use
aspect == aspect_type.ASPECT_NAMEinsteadReplace Yoda condition with
aspect == aspect_type.ASPECT_NAME(SIM300)
724-724: Use
enumerate()for index variableentities_yieldedinforloop(SIM113)
1180-1180: Use of
functools.lru_cacheorfunctools.cacheon methods can lead to memory leaks(B019)
1331-1331: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1350-1350: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1354-1363: Use format specifiers instead of percent format
Replace with format specifiers
(UP031)
1383-1383: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1387-1403: Use format specifiers instead of percent format
Replace with format specifiers
(UP031)
1422-1422: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1423-1423: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1427-1443: Use format specifiers instead of percent format
Replace with format specifiers
(UP031)
1463-1463: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
metadata-ingestion/tests/unit/test_transform_dataset.py
126-126: Do not perform function call
models.StatusClassin argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable(B008)
LanguageTool
metadata-ingestion/docs/transformer/dataset_transformer.md
[uncategorized] ~24-~24: Between ‘whether’ and ‘should’ a pronoun, proper noun or plural noun seems to be missing.
Context: ... | | bool |true| Whether should be consider a user or not. Iffalset...(LACK_OF_PRP_OR_NOUN_FROM_WHETHER_SHOULD)
[grammar] ~24-~24: There may an error in the verb form ‘be consider’.
Context: ... | bool |true| Whether should be consider a user or not. Iffalsethen consider...(MD_BE_NON_VBP)
[uncategorized] ~46-~46: A comma may be missing after the conjunctive/linking adverb ‘Hence’.
Context: ...attern will be converted into an owner. Hence users[email protected]and `[email protected]...(SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA)
[uncategorized] ~59-~59: Possible missing comma found.
Context: ...main: "email.com" ``` - Add owners, however owner type and owner type urn wanted to...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~68-~68: The verb form ‘needs’ does not seem to match the subject ‘characters’.
Context: ...Add owners, however some tag characters needs to replace with some other characters b...(SUBJECT_VERB_AGREEMENT_PLURAL)
[grammar] ~80-~80: It appears that the verb form is incorrect or a word is missing.
Context: ...owners, however owner type also need to extracted from tag pattern. For example: from tag...(HAS_TO_APPROVED_BY)
[misspelling] ~96-~96: Use “an” instead of ‘a’ if the following word starts with a vowel sound, e.g. ‘an article’, ‘an hour’.
Context: ...om the Owner URN(s) | Matches against a Onwer URN and remove the matching part ...(EN_A_VS_AN)
[grammar] ~128-~128: In this context, ‘type’ should agree in number with the noun after ‘of’.
Context: ... string | "DATAOWNER" | ownership type of the owners (either as enum or ownership type urn) ...(TYPE_OF_PLURAL)
[style] ~153-~153: To form a complete sentence, be sure to include a subject.
Context: ...R" ```simple_add_dataset_ownershipcan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~179-~179: Possible missing comma found.
Context: ...ystem__producer" ``` - Add owners, however keep the owners available for the datas...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~197-~197: In this context, ‘type’ should agree in number with the noun after ‘of’.
Context: ... | "DATAOWNER" | ownership type of the owners (either as enum or ownership type urn) ...(TYPE_OF_PLURAL)
[style] ~216-~216: To form a complete sentence, be sure to include a subject.
Context: ... ```pattern_add_dataset_ownershipcan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~242-~242: Possible missing comma found.
Context: ...system__producer" ``` - Add owner, however keep the owners available for the datas...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~256-~256: Possible missing comma found.
Context: ...clear existing owners sent by ingestion source we can use the `simple_remove_dataset_o...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~271-~271: A comma may be missing after the conjunctive/linking adverb ‘Currently’.
Context: ... | Which field to extract tag from. Currently onlyurnis supported. | | `extract_...(SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA)
[style] ~316-~316: To form a complete sentence, be sure to include a subject.
Context: ...egacy" ```simple_add_dataset_tagscan be configured in below different way ...(MISSING_IT_THERE)
[style] ~370-~370: To form a complete sentence, be sure to include a subject.
Context: ...ion"] ```pattern_add_dataset_tagscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~383-~383: Possible missing comma found.
Context: ...eedsDocumentation"] ``` - Add tags, however overwrite the tags available for the da...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~394-~394: Possible missing comma found.
Context: ...eedsDocumentation"] ``` - Add tags, however keep the tags available for the dataset...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~449-~449: To form a complete sentence, be sure to include a subject.
Context: ...lling-the-package).add_dataset_tagscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~467-~467: Possible missing comma found.
Context: ...e>.<your_function>" ``` - Add tags, however keep the tags available for the dataset...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~530-~530: To form a complete sentence, be sure to include a subject.
Context: ...e transform.set_dataset_browse_pathcan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~541-~541: Possible missing comma found.
Context: ...ATASET_PARTS ``` - Add browsePath, however overwrite the browsePath available for ...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~582-~582: To form a complete sentence, be sure to include a subject.
Context: ...ress" ```simple_add_dataset_termscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~594-~594: Possible missing comma found.
Context: ...ssaryTerm:Address" ``` - Add terms, however overwrite the terms available for the d...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~604-~604: Possible missing comma found.
Context: ...ssaryTerm:Address" ``` - Add terms, however keep the terms available for the datase...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~635-~635: To form a complete sentence, be sure to include a subject.
Context: ...de"] ```pattern_add_dataset_termscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~649-~649: Possible missing comma found.
Context: ...Term:PostalCode"] ``` - Add terms, however overwrite the terms available for the d...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~660-~660: Possible missing comma found.
Context: ...yTerm:PostalCode"] ``` - Add terms, however keep the terms available for the datase...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~743-~743: To form a complete sentence, be sure to include a subject.
Context: ...```pattern_add_dataset_schema_termscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~756-~756: Possible missing comma found.
Context: ...lossaryTerm:Name"] ``` - Add terms, however overwrite the terms available for the d...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~767-~767: Possible missing comma found.
Context: ...lossaryTerm:Name"] ``` - Add terms, however keep the terms available for the datase...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~803-~803: To form a complete sentence, be sure to include a subject.
Context: ... ```pattern_add_dataset_schema_tagscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~816-~816: Possible missing comma found.
Context: ...eedsDocumentation"] ``` - Add tags, however overwrite the tags available for the da...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~827-~827: Possible missing comma found.
Context: ...eedsDocumentation"] ``` - Add tags, however keep the tags available for the dataset...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~847-~847: Possible missing comma found.
Context: ...field is a dictionary of string values. Note in case of any key collision, the value...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~859-~859: To form a complete sentence, be sure to include a subject.
Context: ... ```simple_add_dataset_propertiescan be configured in below different way ...(MISSING_IT_THERE)
[style] ~926-~926: To form a complete sentence, be sure to include a subject.
Context: ...perties ```add_dataset_propertiescan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~999-~999: Possible missing preposition found.
Context: ...existingandsemantics`, please refer section [Relationship Between replace_existing ...(AI_HYDRA_LEO_MISSING_TO)
[uncategorized] ~1003-~1003: Possible missing article found.
Context: ... we’d like to add a series of domain to dataset, in this case you can use `simple_add_d...(AI_HYDRA_LEO_MISSING_A)
[style] ~1018-~1018: To form a complete sentence, be sure to include a subject.
Context: ...ering ```simple_add_dataset_domaincan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~1020-~1020: Possible missing comma found.
Context: ...in below different way - Add domains, however replace existing domains sent by ingest...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~1030-~1030: Possible missing comma found.
Context: ...urn:li:domain:hr" ``` - Add domains, however overwrite the domains available for the...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~1055-~1055: The past participle is required after “to be”.
Context: ...le domain name or domain urn need to be apply on matching dataset urn. | | `replace_e...(BE_VBP_IN)
[style] ~1077-~1077: To form a complete sentence, be sure to include a subject.
Context: ...e"] ```pattern_add_dataset_domaincan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~1090-~1090: Possible missing comma found.
Context: ...omain:finance"] ``` - Add domains, however overwrite the domains available for the...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~1101-~1101: Possible missing comma found.
Context: ...omain:finance"] ``` - Add domains, however keep the domains available for the data...(AI_HYDRA_LEO_MISSING_COMMA)
[duplication] ~1133-~1133: Possible typo: you repeated a word
Context: ... For example, instead of using the tag URN urn:li:tag:NeedsDocumentation, you should s...(ENGLISH_WORD_REPEAT_RULE)
[style] ~1144-~1144: To form a complete sentence, be sure to include a subject.
Context: ...n" ```domain_mapping_based_on_tagscan be configured in below different way -...(MISSING_IT_THERE)
[typographical] ~1237-~1237: The word “however” is an adverb that can’t be used like a conjunction, and therefore needs to be separated from the sentence.
Context: ...ontext ofsimple_add_dataset_ownership, however it is applicable for all dataset transf...(HOWEVER_SENTENCE)
[uncategorized] ~1364-~1364: Possible missing comma found.
Context: ..._dict) return cls(config, ctx) ``` Next we need to tell the helper classes whic...(AI_HYDRA_LEO_MISSING_COMMA)
[typographical] ~1374-~1374: Consider adding a comma after ‘Finally’ for more clarity.
Context: ...lf) -> str: return "ownership" ``` Finally we need to implement the `transform_asp...(RB_LY_COMMA)
Markdownlint
metadata-ingestion/docs/transformer/dataset_transformer.md
4-4: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
7-7: Expected: 0 or 2; Actual: 95
Trailing spaces(MD009, no-trailing-spaces)
113-113: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
123-123: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
153-153: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
192-192: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
216-216: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
288-288: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
316-316: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
370-370: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
449-449: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
518-518: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
530-530: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
568-568: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
582-582: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
635-635: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
743-743: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
803-803: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
859-859: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
926-926: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
991-991: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1018-1018: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1051-1051: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1059-1059: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1063-1063: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1077-1077: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1192-1192: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1248-1248: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1255-1255: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1261-1261: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1268-1268: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1302-1302: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1402-1402: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1404-1404: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1426-1426: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1427-1427: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1417-1417: Column: 1
Hard tabs(MD010, no-hard-tabs)
1517-1517: Column: 1
Hard tabs(MD010, no-hard-tabs)
95-95: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
152-152: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
294-294: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
786-786: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
962-962: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
979-979: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
990-990: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1017-1017: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1113-1113: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1114-1114: Expected: 1; Actual: 3
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1143-1143: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1270-1270: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1271-1271: Expected: 1; Actual: 3
Multiple consecutive blank lines(MD012, no-multiple-blanks)
4-4: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
19-19: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
20-20: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
20-20: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
89-89: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
90-90: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
90-90: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
107-107: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
108-108: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
108-108: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
123-123: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
123-123: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
124-124: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
124-124: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
192-192: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
193-193: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
193-193: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
255-255: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
267-267: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
267-267: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
268-268: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
268-268: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
295-295: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
296-296: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
296-296: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
348-348: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
348-348: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
349-349: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
349-349: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
405-405: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
405-405: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
406-406: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
406-406: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
475-475: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
475-475: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
476-476: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
476-476: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
560-560: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
561-561: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
561-561: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
615-615: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
616-616: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
616-616: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
672-672: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
673-673: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
721-721: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
722-722: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
722-722: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
778-778: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
778-778: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
779-779: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
779-779: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
838-838: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
838-838: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
839-839: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
839-839: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
892-892: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
893-893: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
893-893: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
955-955: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
956-956: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
956-956: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
973-973: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
974-974: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
974-974: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
991-991: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
992-992: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
992-992: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1051-1051: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1052-1052: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1052-1052: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1115-1115: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1116-1116: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1167-1167: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1168-1168: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1168-1168: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1186-1186: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1187-1187: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1187-1187: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1206-1206: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1207-1207: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1207-1207: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1236-1236: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
4-4: null
Multiple top-level headings in the same document(MD025, single-title, single-h1)
51-51: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
58-58: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
60-60: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
67-67: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
69-69: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
79-79: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
81-81: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
122-122: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
156-156: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
166-166: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
168-168: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
178-178: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
180-180: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
219-219: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
229-229: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
231-231: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
241-241: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
243-243: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
319-319: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
327-327: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
329-329: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
337-337: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
339-339: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
347-347: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
373-373: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
382-382: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
384-384: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
393-393: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
395-395: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
404-404: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
446-446: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
452-452: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
458-458: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
460-460: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
466-466: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
468-468: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
474-474: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
489-489: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
498-498: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
504-504: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
508-508: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
515-515: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
520-520: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
527-527: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
533-533: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
540-540: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
542-542: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
549-549: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
551-551: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
585-585: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
593-593: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
595-595: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
603-603: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
605-605: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
638-638: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
648-648: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
650-650: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
659-659: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
661-661: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
700-700: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
709-709: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
711-711: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
746-746: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
755-755: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
757-757: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
766-766: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
768-768: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
777-777: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
806-806: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
815-815: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
817-817: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
826-826: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
828-828: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
837-837: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
862-862: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
870-870: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
872-872: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
880-880: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
882-882: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
929-929: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
936-936: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
938-938: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
945-945: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
947-947: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
981-981: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1008-1008: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1021-1021: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1029-1029: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1031-1031: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1039-1039: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1041-1041: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1080-1080: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1089-1089: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1091-1091: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1100-1100: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1102-1102: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1147-1147: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1155-1155: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1157-1157: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1233-1233: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
43-43: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
289-289: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
347-347: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
404-404: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
474-474: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
484-484: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
699-699: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
710-710: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
710-710: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
777-777: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
837-837: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1245-1245: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1251-1251: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1258-1258: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1264-1264: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
17-17: null
Spaces inside link text(MD039, no-space-in-links)
17-17: Expected: leading_and_trailing; Actual: leading_only; Missing trailing pipe
Table pipe style(MD055, table-pipe-style)
Additional comments not posted (8)
metadata-ingestion/docs/transformer/dataset_transformer.md (3)
672-672: Add blank line before the heading.Headings should be surrounded by blank lines for better readability.
## Tags to Term MappingTools
Markdownlint
672-672: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
672-672: Fix heading level consistency.Ensure heading levels are consistent throughout the document.
## Tags to Term MappingTools
Markdownlint
672-672: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
674-679: Fix grammatical issues in the configuration table.Correct the grammatical errors for clarity.
| `tags` | ✅ | List[str] | | List of tag names based on which terms will be created and associated with the dataset. | | `semantics` | | enum | "OVERWRITE" | Determines whether to OVERWRITE or PATCH the terms associated with the dataset on DataHub GMS. |Likely invalid or redundant comment.
metadata-ingestion/tests/unit/test_transform_dataset.py (5)
73-76: Add missing import forTagsToTermTransformer.The
TagsToTermTransformerclass is mentioned in the AI-generated summary but not imported here. Ensure it is imported if it is used in the tests.from datahub.ingestion.transformer.dataset_transformer import ( DatasetTransformer, TagTransformer, TagsToTermTransformer, # Add this line )
92-92: Add missing import forTagsToTermMapperConfig.The
TagsToTermMapperConfigclass is mentioned in the AI-generated summary but not imported here. Ensure it is imported if it is used in the tests.from datahub.ingestion.transformer.tags_to_terms import TagsToTermMapper, TagsToTermMapperConfig
1898-1904: LGTM!The changes to include
TagTransformerin the functionrun_dataset_transformer_pipelineare well-structured and correctly typed.
3662-3760: LGTM!The test function
test_tags_to_terms_transformationis well-structured and covers the transformation logic comprehensively.
3763-3869: LGTM!The test functions
test_tags_to_terms_with_no_matching_terms,test_tags_to_terms_with_missing_tags, andtest_tags_to_terms_with_partial_matchare well-structured and cover the transformation logic comprehensively.
sagar-salvi-apptware
force-pushed
the
fix/ING-647/tags-to-term-transformer
branch
from
12fd649 to
15e544c
Compare
There was a problem hiding this comment.
Actionable comments posted: 7
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (3)
- metadata-ingestion/docs/transformer/dataset_transformer.md (2 hunks)
- metadata-ingestion/src/datahub/ingestion/graph/client.py (1 hunks)
- metadata-ingestion/tests/unit/test_transform_dataset.py (4 hunks)
Additional context used
Ruff
metadata-ingestion/src/datahub/ingestion/graph/client.py
317-317: Yoda conditions are discouraged, use
aspect == aspect_type.ASPECT_NAMEinsteadReplace Yoda condition with
aspect == aspect_type.ASPECT_NAME(SIM300)
724-724: Use
enumerate()for index variableentities_yieldedinforloop(SIM113)
1180-1180: Use of
functools.lru_cacheorfunctools.cacheon methods can lead to memory leaks(B019)
1331-1331: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1350-1350: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1354-1363: Use format specifiers instead of percent format
Replace with format specifiers
(UP031)
1383-1383: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1387-1403: Use format specifiers instead of percent format
Replace with format specifiers
(UP031)
1422-1422: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1423-1423: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
1427-1443: Use format specifiers instead of percent format
Replace with format specifiers
(UP031)
1463-1463: Do not use mutable data structures for argument defaults
Replace with
None; initialize within function(B006)
metadata-ingestion/tests/unit/test_transform_dataset.py
126-126: Do not perform function call
models.StatusClassin argument defaults; instead, perform the call within the function, or read the default from a module-level singleton variable(B008)
LanguageTool
metadata-ingestion/docs/transformer/dataset_transformer.md
[uncategorized] ~24-~24: Between ‘whether’ and ‘should’ a pronoun, proper noun or plural noun seems to be missing.
Context: ... | | bool |true| Whether should be consider a user or not. Iffalset...(LACK_OF_PRP_OR_NOUN_FROM_WHETHER_SHOULD)
[grammar] ~24-~24: There may an error in the verb form ‘be consider’.
Context: ... | bool |true| Whether should be consider a user or not. Iffalsethen consider...(MD_BE_NON_VBP)
[uncategorized] ~46-~46: A comma may be missing after the conjunctive/linking adverb ‘Hence’.
Context: ...attern will be converted into an owner. Hence users[email protected]and `[email protected]...(SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA)
[uncategorized] ~50-~50: Possible missing comma found.
Context: ...as owners. ### Examples - Add owners, however owner should be considered as group and...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~59-~59: Possible missing comma found.
Context: ...main: "email.com" ``` - Add owners, however owner type and owner type urn wanted to...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~68-~68: The verb form ‘needs’ does not seem to match the subject ‘characters’.
Context: ...Add owners, however some tag characters needs to replace with some other characters b...(SUBJECT_VERB_AGREEMENT_PLURAL)
[uncategorized] ~80-~80: Possible missing comma found.
Context: ... "__": "_" ``` - Add owners, however owner type also need to extracted from ...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~80-~80: It appears that the verb form is incorrect or a word is missing.
Context: ...owners, however owner type also need to extracted from tag pattern. For example: from tag...(HAS_TO_APPROVED_BY)
[misspelling] ~96-~96: Use “an” instead of ‘a’ if the following word starts with a vowel sound, e.g. ‘an article’, ‘an hour’.
Context: ...om the Owner URN(s) | Matches against a Onwer URN and remove the matching part ...(EN_A_VS_AN)
[grammar] ~128-~128: In this context, ‘type’ should agree in number with the noun after ‘of’.
Context: ... string | "DATAOWNER" | ownership type of the owners (either as enum or ownership type urn) ...(TYPE_OF_PLURAL)
[style] ~153-~153: To form a complete sentence, be sure to include a subject.
Context: ...R" ```simple_add_dataset_ownershipcan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~155-~155: Possible missing comma found.
Context: ... in below different way - Add owners, however replace existing owners sent by ingesti...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~179-~179: Possible missing comma found.
Context: ...ystem__producer" ``` - Add owners, however keep the owners available for the datas...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~197-~197: In this context, ‘type’ should agree in number with the noun after ‘of’.
Context: ... | "DATAOWNER" | ownership type of the owners (either as enum or ownership type urn) ...(TYPE_OF_PLURAL)
[style] ~216-~216: To form a complete sentence, be sure to include a subject.
Context: ... ```pattern_add_dataset_ownershipcan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~218-~218: Possible missing comma found.
Context: ...d in below different way - Add owner, however replace existing owner sent by ingestio...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~271-~271: A comma may be missing after the conjunctive/linking adverb ‘Currently’.
Context: ... | Which field to extract tag from. Currently onlyurnis supported. | | `extract_...(SENT_START_CONJUNCTIVE_LINKING_ADVERB_COMMA)
[style] ~316-~316: To form a complete sentence, be sure to include a subject.
Context: ...egacy" ```simple_add_dataset_tagscan be configured in below different way ...(MISSING_IT_THERE)
[style] ~370-~370: To form a complete sentence, be sure to include a subject.
Context: ...ion"] ```pattern_add_dataset_tagscan be configured in below different way ...(MISSING_IT_THERE)
[style] ~449-~449: To form a complete sentence, be sure to include a subject.
Context: ...lling-the-package).add_dataset_tagscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~467-~467: Possible missing comma found.
Context: ...e>.<your_function>" ``` - Add tags, however keep the tags available for the dataset...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~530-~530: To form a complete sentence, be sure to include a subject.
Context: ...e transform.set_dataset_browse_pathcan be configured in below different way ...(MISSING_IT_THERE)
[style] ~582-~582: To form a complete sentence, be sure to include a subject.
Context: ...ress" ```simple_add_dataset_termscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~604-~604: Possible missing comma found.
Context: ...ssaryTerm:Address" ``` - Add terms, however keep the terms available for the datase...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~635-~635: To form a complete sentence, be sure to include a subject.
Context: ...de"] ```pattern_add_dataset_termscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~649-~649: Possible missing comma found.
Context: ...Term:PostalCode"] ``` - Add terms, however overwrite the terms available for the d...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~660-~660: Possible missing comma found.
Context: ...yTerm:PostalCode"] ``` - Add terms, however keep the terms available for the datase...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~744-~744: To form a complete sentence, be sure to include a subject.
Context: ...```pattern_add_dataset_schema_termscan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~757-~757: Possible missing comma found.
Context: ...lossaryTerm:Name"] ``` - Add terms, however overwrite the terms available for the d...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~768-~768: Possible missing comma found.
Context: ...lossaryTerm:Name"] ``` - Add terms, however keep the terms available for the datase...(AI_HYDRA_LEO_MISSING_COMMA)
[style] ~804-~804: To form a complete sentence, be sure to include a subject.
Context: ... ```pattern_add_dataset_schema_tagscan be configured in below different way ...(MISSING_IT_THERE)
[style] ~860-~860: To form a complete sentence, be sure to include a subject.
Context: ... ```simple_add_dataset_propertiescan be configured in below different way ...(MISSING_IT_THERE)
[style] ~927-~927: To form a complete sentence, be sure to include a subject.
Context: ...perties ```add_dataset_propertiescan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~1004-~1004: Possible missing article found.
Context: ... we’d like to add a series of domain to dataset, in this case you can use `simple_add_d...(AI_HYDRA_LEO_MISSING_A)
[style] ~1019-~1019: To form a complete sentence, be sure to include a subject.
Context: ...ering ```simple_add_dataset_domaincan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~1021-~1021: Possible missing comma found.
Context: ...in below different way - Add domains, however replace existing domains sent by ingest...(AI_HYDRA_LEO_MISSING_COMMA)
[uncategorized] ~1041-~1041: Possible missing comma found.
Context: ...rn:li:domain:hr" ``` - Add domains, however keep the domains available for the data...(AI_HYDRA_LEO_MISSING_COMMA)
[grammar] ~1056-~1056: The past participle is required after “to be”.
Context: ...le domain name or domain urn need to be apply on matching dataset urn. | | `replace_e...(BE_VBP_IN)
[style] ~1078-~1078: To form a complete sentence, be sure to include a subject.
Context: ...e"] ```pattern_add_dataset_domaincan be configured in below different way ...(MISSING_IT_THERE)
[uncategorized] ~1091-~1091: Possible missing comma found.
Context: ...omain:finance"] ``` - Add domains, however overwrite the domains available for the...(AI_HYDRA_LEO_MISSING_COMMA)
[duplication] ~1134-~1134: Possible typo: you repeated a word
Context: ... For example, instead of using the tag URN urn:li:tag:NeedsDocumentation, you should s...(ENGLISH_WORD_REPEAT_RULE)
[style] ~1145-~1145: To form a complete sentence, be sure to include a subject.
Context: ...n" ```domain_mapping_based_on_tagscan be configured in below different way -...(MISSING_IT_THERE)
[typographical] ~1238-~1238: The word “however” is an adverb that can’t be used like a conjunction, and therefore needs to be separated from the sentence.
Context: ...ontext ofsimple_add_dataset_ownership, however it is applicable for all dataset transf...(HOWEVER_SENTENCE)
[typographical] ~1375-~1375: Consider adding a comma after ‘Finally’ for more clarity.
Context: ...lf) -> str: return "ownership" ``` Finally we need to implement the `transform_asp...(RB_LY_COMMA)
Markdownlint
metadata-ingestion/docs/transformer/dataset_transformer.md
4-4: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
7-7: Expected: 0 or 2; Actual: 95
Trailing spaces(MD009, no-trailing-spaces)
113-113: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
123-123: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
153-153: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
192-192: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
216-216: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
288-288: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
316-316: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
370-370: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
449-449: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
518-518: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
530-530: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
568-568: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
582-582: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
635-635: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
744-744: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
804-804: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
860-860: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
927-927: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
992-992: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1019-1019: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1052-1052: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1060-1060: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1064-1064: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1078-1078: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1193-1193: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1249-1249: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1256-1256: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1262-1262: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1269-1269: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1303-1303: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1403-1403: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1405-1405: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1427-1427: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1428-1428: Expected: 0 or 2; Actual: 1
Trailing spaces(MD009, no-trailing-spaces)
1418-1418: Column: 1
Hard tabs(MD010, no-hard-tabs)
1518-1518: Column: 1
Hard tabs(MD010, no-hard-tabs)
95-95: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
152-152: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
294-294: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
787-787: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
963-963: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
980-980: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
991-991: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1018-1018: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1114-1114: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1115-1115: Expected: 1; Actual: 3
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1144-1144: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1271-1271: Expected: 1; Actual: 2
Multiple consecutive blank lines(MD012, no-multiple-blanks)
1272-1272: Expected: 1; Actual: 3
Multiple consecutive blank lines(MD012, no-multiple-blanks)
4-4: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
19-19: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
20-20: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
20-20: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
89-89: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
90-90: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
90-90: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
107-107: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
108-108: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
108-108: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
123-123: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
123-123: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
124-124: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
124-124: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
192-192: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
193-193: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
193-193: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
255-255: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
267-267: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
267-267: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
268-268: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
268-268: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
295-295: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
296-296: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
296-296: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
348-348: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
348-348: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
349-349: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
349-349: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
405-405: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
405-405: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
406-406: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
406-406: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
475-475: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
475-475: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
476-476: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
476-476: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
560-560: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
561-561: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
561-561: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
615-615: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
616-616: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
616-616: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
672-672: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
673-673: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
722-722: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
723-723: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
723-723: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
779-779: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
779-779: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
780-780: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
780-780: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
839-839: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
839-839: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
840-840: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
840-840: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
893-893: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
894-894: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
894-894: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
956-956: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
957-957: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
957-957: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
974-974: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
975-975: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
975-975: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
992-992: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
993-993: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
993-993: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1052-1052: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1053-1053: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1053-1053: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1116-1116: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1117-1117: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1168-1168: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1169-1169: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1169-1169: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1187-1187: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1188-1188: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1188-1188: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1207-1207: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1208-1208: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1208-1208: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
1237-1237: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
4-4: null
Multiple top-level headings in the same document(MD025, single-title, single-h1)
51-51: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
58-58: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
60-60: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
67-67: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
69-69: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
79-79: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
81-81: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
122-122: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
156-156: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
166-166: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
168-168: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
178-178: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
180-180: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
219-219: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
229-229: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
231-231: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
241-241: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
243-243: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
319-319: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
327-327: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
329-329: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
337-337: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
339-339: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
347-347: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
373-373: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
382-382: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
384-384: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
393-393: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
395-395: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
404-404: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
446-446: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
452-452: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
458-458: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
460-460: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
466-466: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
468-468: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
474-474: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
489-489: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
498-498: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
504-504: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
508-508: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
515-515: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
520-520: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
527-527: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
533-533: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
540-540: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
542-542: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
549-549: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
551-551: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
585-585: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
593-593: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
595-595: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
603-603: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
605-605: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
638-638: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
648-648: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
650-650: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
659-659: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
661-661: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
700-700: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
709-709: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
711-711: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
747-747: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
756-756: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
758-758: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
767-767: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
769-769: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
778-778: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
807-807: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
816-816: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
818-818: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
827-827: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
829-829: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
838-838: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
863-863: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
871-871: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
873-873: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
881-881: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
883-883: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
930-930: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
937-937: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
939-939: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
946-946: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
948-948: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
982-982: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1009-1009: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1022-1022: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1030-1030: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1032-1032: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1040-1040: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1042-1042: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1081-1081: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1090-1090: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1092-1092: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1101-1101: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1103-1103: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1148-1148: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1156-1156: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1158-1158: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
1234-1234: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
43-43: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
289-289: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
347-347: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
404-404: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
474-474: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
484-484: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
699-699: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
710-710: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
710-710: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
778-778: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
838-838: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1246-1246: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1252-1252: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1259-1259: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
1265-1265: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
11-11: null
Spaces inside link text(MD039, no-space-in-links)
13-13: null
Spaces inside link text(MD039, no-space-in-links)
17-17: null
Spaces inside link text(MD039, no-space-in-links)
17-17: Expected: leading_and_trailing; Actual: leading_only; Missing trailing pipe
Table pipe style(MD055, table-pipe-style)
Additional comments not posted (12)
metadata-ingestion/src/datahub/ingestion/graph/client.py (1)
1281-1293: Add logging for the remove_tag operation.Adding logging helps in tracing the operations performed by the function.
def remove_tag(self, tag_urn: str, resource_urn: str) -> bool: graph_query = f""" mutation removeTag {{ removeTag( input: {{ tagUrn: "{tag_urn}", resource_urn: "{resource_urn}" }}) }} """ res = self.execute_graphql(query=graph_query) + logger.info(f"Removing tag {tag_urn} from resource {resource_urn}") return res["removeTag"]metadata-ingestion/docs/transformer/dataset_transformer.md (5)
13-13: Entry forTags to Term Mappinglooks good.The addition of the new transformer to the table is correct and follows the existing pattern.
Tools
Markdownlint
13-13: null
Spaces inside link text(MD039, no-space-in-links)
684-684: Rewrite for clarity.Rewrite the sentence for better clarity and readability.
- When specifying tags in the configuration, use the tag's simple name rather than the full tag URN. + When specifying tags in the configuration, use the tag's simple name instead of the full tag URN.Likely invalid or redundant comment.
697-697: Complete the sentence.Add a subject to form a complete sentence.
- `tags_to_term` can be configured in below different way + The `tags_to_term` transformer can be configured in the following ways:Likely invalid or redundant comment.
699-699: Fix example heading.The example heading is incorrect for the context of tags to terms mapping.
- Add domains based on tags, however overwrite the domains available for the dataset on DataHub GMS + Add terms based on tags, however overwrite the terms available for the dataset on DataHub GMSLikely invalid or redundant comment.
Tools
Markdownlint
699-699: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
710-710: Fix example heading.The example heading is incorrect for the context of tags to terms mapping.
- Add domains based on tags, however keep the domains available for the dataset on DataHub GMS + Add terms based on tags, however keep the terms available for the dataset on DataHub GMSLikely invalid or redundant comment.
Tools
Markdownlint
710-710: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
710-710: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
metadata-ingestion/tests/unit/test_transform_dataset.py (6)
3763-3797: LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
test_tags_to_terms_with_no_matching_termsare correctly implemented.Verification successful
Function usage verified successfully.
The function
test_tags_to_terms_with_no_matching_termsis defined but not called elsewhere in the codebase, which is expected for a test case function.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify all function calls to `test_tags_to_terms_with_no_matching_terms`. # Test: Search for the function usage. Expect: Only occurances of the new signature. rg --type python $'test_tags_to_terms_with_no_matching_terms'Length of output: 91
Script:
#!/bin/bash # Description: Verify all function calls to `test_tags_to_terms_with_no_matching_terms`. # Correct the file type and scope the search to the test directory rg 'test_tags_to_terms_with_no_matching_terms' --glob '*.py' metadata-ingestion/tests/unit/Length of output: 214
3800-3828: LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
test_tags_to_terms_with_missing_tagsare correctly implemented.
3831-3869: LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
test_tags_to_terms_with_partial_matchare correctly implemented.Verification successful
Function usage verified.
The function
test_tags_to_terms_with_partial_matchis defined correctly in the codebase and, as a test function, does not need to be called elsewhere.
- No additional occurrences of the function were found.
Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify all function calls to `test_tags_to_terms_with_partial_match`. # Test: Search for the function usage. Expect: Only occurances of the new signature. rg --type python $'test_tags_to_terms_with_partial_match'Length of output: 87
Script:
#!/bin/bash # Description: Verify all function calls to `test_tags_to_terms_with_partial_match`. # Test: Search for the function usage. Expect: Only occurrences of the new signature. rg --type py $'test_tags_to_terms_with_partial_match'Length of output: 171
3662-3760: LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
test_tags_to_terms_transformationare correctly implemented.Verification successful
Function usage verified
The function
test_tags_to_terms_transformationis only defined in the test file and is not called or referenced elsewhere in the codebase.
metadata-ingestion/tests/unit/test_transform_dataset.py:def test_tags_to_terms_transformation(mock_datahub_graph):Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify all function calls to `test_tags_to_terms_transformation`. # Test: Search for the function usage. Expect: Only occurances of the new signature. rg --type python $'test_tags_to_terms_transformation'Length of output: 83
Script:
#!/bin/bash # Description: Verify all function calls to `test_tags_to_terms_transformation`. # Test: Search for the function usage. Expect: Only occurrences of the new signature. rg -t py 'test_tags_to_terms_transformation'Length of output: 159
1898-1905: LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
run_dataset_transformer_pipelineare correctly implemented.Verification successful
Function usage verified.
All occurrences of
run_dataset_transformer_pipelineare within themetadata-ingestion/tests/unit/test_transform_dataset.pyfile, indicating that the function is used exclusively for testing purposes. No issues found.
metadata-ingestion/tests/unit/test_transform_dataset.pyScripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify all function calls to `run_dataset_transformer_pipeline`. # Test: Search for the function usage. Expect: Only occurances of the new signature. rg --type python $'run_dataset_transformer_pipeline'Length of output: 82
Script:
#!/bin/bash # Description: Verify all function calls to `run_dataset_transformer_pipeline`. # Test: Search for the function usage. Expect: Only occurrences of the new signature. rg --type py $'run_dataset_transformer_pipeline'Length of output: 3645
3662-3760: LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
run_pattern_dataset_schema_terms_transformation_semanticsare correctly implemented.Also applies to: 3763-3797, 3800-3828, 3831-3869, 1898-1905
Verification successful
Function usage verified.
The function
run_pattern_dataset_schema_terms_transformation_semanticsis used only within themetadata-ingestion/tests/unit/test_transform_dataset.pyfile, indicating it is correctly implemented and tested.
metadata-ingestion/tests/unit/test_transform_dataset.py: Verified function usage.Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify all function calls to `run_pattern_dataset_schema_terms_transformation_semantics`. # Test: Search for the function usage. Expect: Only occurances of the new signature. rg --type python $'run_pattern_dataset_schema_terms_transformation_semantics'Length of output: 107
Script:
#!/bin/bash # Description: Verify all function calls to `run_pattern_dataset_schema_terms_transformation_semantics`. # Test: Search for the function usage. Expect: Only occurrences of the new signature. rg -t py 'run_pattern_dataset_schema_terms_transformation_semantics'Length of output: 443
| def remove_tag(self, tag_urn: str, resource_urn: str) -> bool: | ||
| graph_query = f""" | ||
| mutation removeTag {{ | ||
| removeTag( | ||
| input: {{ | ||
| tagUrn: "{tag_urn}", | ||
| resourceUrn: "{resource_urn}" | ||
| }}) | ||
| }} | ||
| """ | ||
|
|
||
| res = self.execute_graphql(query=graph_query) | ||
| return res["removeTag"] |
There was a problem hiding this comment.
Add error handling for the GraphQL response.
The function should handle potential errors in the GraphQL response to ensure robustness.
def remove_tag(self, tag_urn: str, resource_urn: str) -> bool:
graph_query = f"""
mutation removeTag {{
removeTag(
input: {{
tagUrn: "{tag_urn}",
resourceUrn: "{resource_urn}"
}})
}}
"""
res = self.execute_graphql(query=graph_query)
+ if "errors" in res:
+ logger.error(f"Failed to remove tag: {res['errors']}")
+ return False
return res["removeTag"]Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def remove_tag(self, tag_urn: str, resource_urn: str) -> bool: | |
| graph_query = f""" | |
| mutation removeTag {{ | |
| removeTag( | |
| input: {{ | |
| tagUrn: "{tag_urn}", | |
| resourceUrn: "{resource_urn}" | |
| }}) | |
| }} | |
| """ | |
| res = self.execute_graphql(query=graph_query) | |
| return res["removeTag"] | |
| def remove_tag(self, tag_urn: str, resource_urn: str) -> bool: | |
| graph_query = f""" | |
| mutation removeTag {{ | |
| removeTag( | |
| input: {{ | |
| tagUrn: "{tag_urn}", | |
| resourceUrn: "{resource_urn}" | |
| }}) | |
| }} | |
| """ | |
| res = self.execute_graphql(query=graph_query) | |
| if "errors" in res: | |
| logger.error(f"Failed to remove tag: {res['errors']}") | |
| return False | |
| return res["removeTag"] |
| The `tags_to_term` transformer can be configured in the following ways: | ||
|
|
||
| - Add terms based on tags, however overwrite the terms available for the dataset on DataHub GMS | ||
| ```yaml |
There was a problem hiding this comment.
Add blank lines around fenced code blocks.
Fenced code blocks should be surrounded by blank lines for better readability.
700a701
+
709a710
+Committable suggestion was skipped due to low confidence.
Tools
Markdownlint
700-700: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
| - "example3" | ||
| ``` | ||
| - Add terms based on tags, however keep the terms available for the dataset on DataHub GMS | ||
| ```yaml |
There was a problem hiding this comment.
Add blank lines around fenced code blocks.
Fenced code blocks should be surrounded by blank lines for better readability.
711a712
+
720a721
+Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ```yaml | |
| ```yaml | |
Tools
Markdownlint
711-711: null
Fenced code blocks should be surrounded by blank lines(MD031, blanks-around-fences)
| ## Tags to Term Mapping | ||
| ### Config Details |
There was a problem hiding this comment.
Add blank lines around headings.
Headings should be surrounded by blank lines for better readability.
672a673
+
673a674
+Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ## Tags to Term Mapping | |
| ### Config Details | |
| ## Tags to Term Mapping | |
| ### Config Details |
Tools
Markdownlint
672-672: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
673-673: Expected: 1; Actual: 0; Above
Headings should be surrounded by blank lines(MD022, blanks-around-headings)
|
|
||
| When specifying tags in the configuration, use the tag's simple name rather than the full tag URN. | ||
|
|
||
| For example, instead of using the tag URN `urn:li:tag:snowflakedb.snowflakeschema.tag_name:tag_value`, you should specify just the tag name `tag_name` in the mapping configuration. |
There was a problem hiding this comment.
Rewrite for clarity.
Rewrite the sentence for better clarity and readability.
- For example, instead of using the tag URN `urn:li:tag:snowflakedb.snowflakeschema.tag_name:tag_value`, you should specify just the tag name `tag_name` in the mapping configuration.
+ For example, instead of using the tag URN `urn:li:tag:snowflakedb.snowflakeschema.tag_name:tag_value`, specify only the tag name `tag_name` in the mapping configuration.Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| For example, instead of using the tag URN `urn:li:tag:snowflakedb.snowflakeschema.tag_name:tag_value`, you should specify just the tag name `tag_name` in the mapping configuration. | |
| For example, instead of using the tag URN `urn:li:tag:snowflakedb.snowflakeschema.tag_name:tag_value`, specify only the tag name `tag_name` in the mapping configuration. |
|
|
||
| <br/> | ||
|
|
||
| The `tags_to_term` transformer is designed to map specific tags to glossary terms within DataHub. It takes a configuration of tags that should be translated into corresponding glossary terms. This transformer can apply these mappings to any tags found either at the column level of a dataset or at the dataset top level. |
There was a problem hiding this comment.
Rewrite for clarity.
Rewrite the sentence for better clarity and readability.
- The `tags_to_term` transformer is designed to map specific tags to glossary terms within DataHub. It takes a configuration of tags that should be translated into corresponding glossary terms. This transformer can apply these mappings to any tags found either at the column level of a dataset or at the dataset top level.
+ The `tags_to_term` transformer maps specific tags to glossary terms within DataHub. It accepts a configuration of tags to be translated into corresponding glossary terms. This transformer can apply these mappings to tags found at both the column level and the dataset top level.Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| The `tags_to_term` transformer is designed to map specific tags to glossary terms within DataHub. It takes a configuration of tags that should be translated into corresponding glossary terms. This transformer can apply these mappings to any tags found either at the column level of a dataset or at the dataset top level. | |
| The `tags_to_term` transformer maps specific tags to glossary terms within DataHub. It accepts a configuration of tags to be translated into corresponding glossary terms. This transformer can apply these mappings to tags found at both the column level and the dataset top level. |
|
|
||
| The `tags_to_term` transformer can be configured in the following ways: | ||
|
|
||
| - Add terms based on tags, however overwrite the terms available for the dataset on DataHub GMS |
There was a problem hiding this comment.
Add blank lines around lists.
Lists should be surrounded by blank lines for better readability.
698a699
+
709a710
+Committable suggestion was skipped due to low confidence.
Tools
Markdownlint
699-699: null
Lists should be surrounded by blank lines(MD032, blanks-around-lists)
Co-authored-by: Aseem Bansal <[email protected]>
…10758) Co-authored-by: Aseem Bansal <[email protected]>
* feat(forms) Handle deleting forms references when hard deleting forms (datahub-project#10820) * refactor(ui): Misc improvements to the setup ingestion flow (ingest uplift 1/2) (datahub-project#10764) Co-authored-by: John Joyce <[email protected]> Co-authored-by: John Joyce <[email protected]> * fix(ingestion/airflow-plugin): pipeline tasks discoverable in search (datahub-project#10819) * feat(ingest/transformer): tags to terms transformer (datahub-project#10758) Co-authored-by: Aseem Bansal <[email protected]> * fix(ingestion/unity-catalog): fixed issue with profiling with GE turned on (datahub-project#10752) Co-authored-by: Aseem Bansal <[email protected]> * feat(forms) Add java SDK for form entity PATCH + CRUD examples (datahub-project#10822) * feat(SDK) Add java SDK for structuredProperty entity PATCH + CRUD examples (datahub-project#10823) * feat(SDK) Add StructuredPropertyPatchBuilder in python sdk and provide sample CRUD files (datahub-project#10824) * feat(forms) Add CRUD endpoints to GraphQL for Form entities (datahub-project#10825) * add flag for includeSoftDeleted in scroll entities API (datahub-project#10831) * feat(deprecation) Return actor entity with deprecation aspect (datahub-project#10832) * feat(structuredProperties) Add CRUD graphql APIs for structured property entities (datahub-project#10826) * add scroll parameters to openapi v3 spec (datahub-project#10833) * fix(ingest): correct profile_day_of_week implementation (datahub-project#10818) * feat(ingest/glue): allow ingestion of empty databases from Glue (datahub-project#10666) Co-authored-by: Harshal Sheth <[email protected]> * feat(cli): add more details to get cli (datahub-project#10815) * fix(ingestion/glue): ensure date formatting works on all platforms for aws glue (datahub-project#10836) * fix(ingestion): fix datajob patcher (datahub-project#10827) * fix(smoke-test): add suffix in temp file creation (datahub-project#10841) * feat(ingest/glue): add helper method to permit user or group ownership (datahub-project#10784) * feat(): Show data platform instances in policy modal if they are set on the policy (datahub-project#10645) Co-authored-by: Hendrik Richert <[email protected]> * docs(patch): add patch documentation for how implementation works (datahub-project#10010) Co-authored-by: John Joyce <[email protected]> * fix(jar): add missing custom-plugin-jar task (datahub-project#10847) * fix(): also check exceptions/stack trace when filtering log messages (datahub-project#10391) Co-authored-by: John Joyce <[email protected]> * docs(): Update posts.md (datahub-project#9893) Co-authored-by: Hyejin Yoon <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * chore(ingest): update acryl-datahub-classify version (datahub-project#10844) * refactor(ingest): Refactor structured logging to support infos, warnings, and failures structured reporting to UI (datahub-project#10828) Co-authored-by: John Joyce <[email protected]> Co-authored-by: Harshal Sheth <[email protected]> * fix(restli): log aspect-not-found as a warning rather than as an error (datahub-project#10834) * fix(ingest/nifi): remove duplicate upstream jobs (datahub-project#10849) * fix(smoke-test): test access to create/revoke personal access tokens (datahub-project#10848) * fix(smoke-test): missing test for move domain (datahub-project#10837) * ci: update usernames to not considered for community (datahub-project#10851) * env: change defaults for data contract visibility (datahub-project#10854) * fix(ingest/tableau): quote special characters in external URL (datahub-project#10842) * fix(smoke-test): fix flakiness of auto complete test * ci(ingest): pin dask dependency for feast (datahub-project#10865) * fix(ingestion/lookml): liquid template resolution and view-to-view cll (datahub-project#10542) * feat(ingest/audit): add client id and version in system metadata props (datahub-project#10829) * chore(ingest): Mypy 1.10.1 pin (datahub-project#10867) * docs: use acryl-datahub-actions as expected python package to install (datahub-project#10852) * docs: add new js snippet (datahub-project#10846) * refactor(ingestion): remove company domain for security reason (datahub-project#10839) * fix(ingestion/spark): Platform instance and column level lineage fix (datahub-project#10843) Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(ingestion/tableau): optionally ingest multiple sites and create site containers (datahub-project#10498) Co-authored-by: Yanik Häni <[email protected]> * fix(ingestion/looker): Add sqlglot dependency and remove unused sqlparser (datahub-project#10874) * fix(manage-tokens): fix manage access token policy (datahub-project#10853) * Batch get entity endpoints (datahub-project#10880) * feat(system): support conditional write semantics (datahub-project#10868) * fix(build): upgrade vercel builds to Node 20.x (datahub-project#10890) * feat(ingest/lookml): shallow clone repos (datahub-project#10888) * fix(ingest/looker): add missing dependency (datahub-project#10876) * fix(ingest): only populate audit stamps where accurate (datahub-project#10604) * fix(ingest/dbt): always encode tag urns (datahub-project#10799) * fix(ingest/redshift): handle multiline alter table commands (datahub-project#10727) * fix(ingestion/looker): column name missing in explore (datahub-project#10892) * fix(lineage) Fix lineage source/dest filtering with explored per hop limit (datahub-project#10879) * feat(conditional-writes): misc updates and fixes (datahub-project#10901) * feat(ci): update outdated action (datahub-project#10899) * feat(rest-emitter): adding async flag to rest emitter (datahub-project#10902) Co-authored-by: Gabe Lyons <[email protected]> * feat(ingest): add snowflake-queries source (datahub-project#10835) * fix(ingest): improve `auto_materialize_referenced_tags_terms` error handling (datahub-project#10906) * docs: add new company to adoption list (datahub-project#10909) * refactor(redshift): Improve redshift error handling with new structured reporting system (datahub-project#10870) Co-authored-by: John Joyce <[email protected]> Co-authored-by: Harshal Sheth <[email protected]> * feat(ui) Finalize support for all entity types on forms (datahub-project#10915) * Index ExecutionRequestResults status field (datahub-project#10811) * feat(ingest): grafana connector (datahub-project#10891) Co-authored-by: Shirshanka Das <[email protected]> Co-authored-by: Harshal Sheth <[email protected]> * fix(gms) Add Form entity type to EntityTypeMapper (datahub-project#10916) * feat(dataset): add support for external url in Dataset (datahub-project#10877) * docs(saas-overview) added missing features to observe section (datahub-project#10913) Co-authored-by: John Joyce <[email protected]> * fix(ingest/spark): Fixing Micrometer warning (datahub-project#10882) * fix(structured properties): allow application of structured properties without schema file (datahub-project#10918) * fix(data-contracts-web) handle other schedule types (datahub-project#10919) * fix(ingestion/tableau): human-readable message for PERMISSIONS_MODE_SWITCHED error (datahub-project#10866) Co-authored-by: Harshal Sheth <[email protected]> * Add feature flag for view defintions (datahub-project#10914) Co-authored-by: Ethan Cartwright <[email protected]> * feat(ingest/BigQuery): refactor+parallelize dataset metadata extraction (datahub-project#10884) * fix(airflow): add error handling around render_template() (datahub-project#10907) * feat(ingestion/sqlglot): add optional `default_dialect` parameter to sqlglot lineage (datahub-project#10830) * feat(mcp-mutator): new mcp mutator plugin (datahub-project#10904) * fix(ingest/bigquery): changes helper function to decode unicode scape sequences (datahub-project#10845) * feat(ingest/postgres): fetch table sizes for profile (datahub-project#10864) * feat(ingest/abs): Adding azure blob storage ingestion source (datahub-project#10813) * fix(ingest/redshift): reduce severity of SQL parsing issues (datahub-project#10924) * fix(build): fix lint fix web react (datahub-project#10896) * fix(ingest/bigquery): handle quota exceeded for project.list requests (datahub-project#10912) * feat(ingest): report extractor failures more loudly (datahub-project#10908) * feat(ingest/snowflake): integrate snowflake-queries into main source (datahub-project#10905) * fix(ingest): fix docs build (datahub-project#10926) * fix(ingest/snowflake): fix test connection (datahub-project#10927) * fix(ingest/lookml): add view load failures to cache (datahub-project#10923) * docs(slack) overhauled setup instructions and screenshots (datahub-project#10922) Co-authored-by: John Joyce <[email protected]> * fix(airflow): Add comma parsing of owners to DataJobs (datahub-project#10903) * fix(entityservice): fix merging sideeffects (datahub-project#10937) * feat(ingest): Support System Ingestion Sources, Show and hide system ingestion sources with Command-S (datahub-project#10938) Co-authored-by: John Joyce <[email protected]> * chore() Set a default lineage filtering end time on backend when a start time is present (datahub-project#10925) Co-authored-by: John Joyce <[email protected]> Co-authored-by: John Joyce <[email protected]> * Added relationships APIs to V3. Added these generic APIs to V3 swagger doc. (datahub-project#10939) * docs: add learning center to docs (datahub-project#10921) * doc: Update hubspot form id (datahub-project#10943) * chore(airflow): add python 3.11 w/ Airflow 2.9 to CI (datahub-project#10941) * fix(ingest/Glue): column upstream lineage between S3 and Glue (datahub-project#10895) * fix(ingest/abs): split abs utils into multiple files (datahub-project#10945) * doc(ingest/looker): fix doc for sql parsing documentation (datahub-project#10883) Co-authored-by: Harshal Sheth <[email protected]> * fix(ingest/bigquery): Adding missing BigQuery types (datahub-project#10950) * fix(ingest/setup): feast and abs source setup (datahub-project#10951) * fix(connections) Harden adding /gms to connections in backend (datahub-project#10942) * feat(siblings) Add flag to prevent combining siblings in the UI (datahub-project#10952) * fix(docs): make graphql doc gen more automated (datahub-project#10953) * feat(ingest/athena): Add option for Athena partitioned profiling (datahub-project#10723) * fix(spark-lineage): default timeout for future responses (datahub-project#10947) * feat(datajob/flow): add environment filter using info aspects (datahub-project#10814) * fix(ui/ingest): correct privilege used to show tab (datahub-project#10483) Co-authored-by: Kunal-kankriya <[email protected]> * feat(ingest/looker): include dashboard urns in browse v2 (datahub-project#10955) * add a structured type to batchGet in OpenAPI V3 spec (datahub-project#10956) * fix(ui): scroll on the domain sidebar to show all domains (datahub-project#10966) * fix(ingest/sagemaker): resolve incorrect variable assignment for SageMaker API call (datahub-project#10965) * fix(airflow/build): Pinning mypy (datahub-project#10972) * Fixed a bug where the OpenAPI V3 spec was incorrect. The bug was introduced in datahub-project#10939. (datahub-project#10974) * fix(ingest/test): Fix for mssql integration tests (datahub-project#10978) * fix(entity-service) exist check correctly extracts status (datahub-project#10973) * fix(structuredProps) casing bug in StructuredPropertiesValidator (datahub-project#10982) * bugfix: use anyOf instead of allOf when creating references in openapi v3 spec (datahub-project#10986) * fix(ui): Remove ant less imports (datahub-project#10988) * feat(ingest/graph): Add get_results_by_filter to DataHubGraph (datahub-project#10987) * feat(ingest/cli): init does not actually support environment variables (datahub-project#10989) * fix(ingest/graph): Update get_results_by_filter graphql query (datahub-project#10991) * feat(ingest/spark): Promote beta plugin (datahub-project#10881) Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(ingest): support domains in meta -> "datahub" section (datahub-project#10967) * feat(ingest): add `check server-config` command (datahub-project#10990) * feat(cli): Make consistent use of DataHubGraphClientConfig (datahub-project#10466) Deprecates get_url_and_token() in favor of a more complete option: load_graph_config() that returns a full DatahubClientConfig. This change was then propagated across previous usages of get_url_and_token so that connections to DataHub server from the client respect the full breadth of configuration specified by DatahubClientConfig. I.e: You can now specify disable_ssl_verification: true in your ~/.datahubenv file so that all cli functions to the server work when ssl certification is disabled. Fixes datahub-project#9705 * fix(ingest/s3): Fixing container creation when there is no folder in path (datahub-project#10993) * fix(ingest/looker): support platform instance for dashboards & charts (datahub-project#10771) * feat(ingest/bigquery): improve handling of information schema in sql parser (datahub-project#10985) * feat(ingest): improve `ingest deploy` command (datahub-project#10944) * fix(backend): allow excluding soft-deleted entities in relationship-queries; exclude soft-deleted members of groups (datahub-project#10920) - allow excluding soft-deleted entities in relationship-queries - exclude soft-deleted members of groups * fix(ingest/looker): downgrade missing chart type log level (datahub-project#10996) * doc(acryl-cloud): release docs for 0.3.4.x (datahub-project#10984) Co-authored-by: John Joyce <[email protected]> Co-authored-by: RyanHolstien <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Pedro Silva <[email protected]> * fix(protobuf/build): Fix protobuf check jar script (datahub-project#11006) * fix(ui/ingest): Support invalid cron jobs (datahub-project#10998) * fix(ingest): fix graph config loading (datahub-project#11002) Co-authored-by: Pedro Silva <[email protected]> * feat(docs): Document __DATAHUB_TO_FILE_ directive (datahub-project#10968) Co-authored-by: Harshal Sheth <[email protected]> * fix(graphql/upsertIngestionSource): Validate cron schedule; parse error in CLI (datahub-project#11011) * feat(ece): support custom ownership type urns in ECE generation (datahub-project#10999) * feat(assertion-v2): changed Validation tab to Quality and created new Governance tab (datahub-project#10935) * fix(ingestion/glue): Add support for missing config options for profiling in Glue (datahub-project#10858) * feat(propagation): Add models for schema field docs, tags, terms (datahub-project#2959) (datahub-project#11016) Co-authored-by: Chris Collins <[email protected]> * docs: standardize terminology to DataHub Cloud (datahub-project#11003) * fix(ingestion/transformer): replace the externalUrl container (datahub-project#11013) * docs(slack) troubleshoot docs (datahub-project#11014) * feat(propagation): Add graphql API (datahub-project#11030) Co-authored-by: Chris Collins <[email protected]> * feat(propagation): Add models for Action feature settings (datahub-project#11029) * docs(custom properties): Remove duplicate from sidebar (datahub-project#11033) * feat(models): Introducing Dataset Partitions Aspect (datahub-project#10997) Co-authored-by: John Joyce <[email protected]> Co-authored-by: John Joyce <[email protected]> * feat(propagation): Add Documentation Propagation Settings (datahub-project#11038) * fix(models): chart schema fields mapping, add dataHubAction entity, t… (datahub-project#11040) * fix(ci): smoke test lint failures (datahub-project#11044) * docs: fix learning center color scheme & typo (datahub-project#11043) * feat: add cloud main page (datahub-project#11017) Co-authored-by: Jay <[email protected]> * feat(restore-indices): add additional step to also clear system metadata service (datahub-project#10662) Co-authored-by: John Joyce <[email protected]> * docs: fix typo (datahub-project#11046) * fix(lint): apply spotless (datahub-project#11050) * docs(airflow): example query to get datajobs for a dataflow (datahub-project#11034) * feat(cli): Add run-id option to put sub-command (datahub-project#11023) Adds an option to assign run-id to a given put command execution. This is useful when transformers do not exist for a given ingestion payload, we can follow up with custom metadata and assign it to an ingestion pipeline. * fix(ingest): improve sql error reporting calls (datahub-project#11025) * fix(airflow): fix CI setup (datahub-project#11031) * feat(ingest/dbt): add experimental `prefer_sql_parser_lineage` flag (datahub-project#11039) * fix(ingestion/lookml): enable stack-trace in lookml logs (datahub-project#10971) * (chore): Linting fix (datahub-project#11015) * chore(ci): update deprecated github actions (datahub-project#10977) * Fix ALB configuration example (datahub-project#10981) * chore(ingestion-base): bump base image packages (datahub-project#11053) * feat(cli): Trim report of dataHubExecutionRequestResult to max GMS size (datahub-project#11051) * fix(ingestion/lookml): emit dummy sql condition for lookml custom condition tag (datahub-project#11008) Co-authored-by: Harshal Sheth <[email protected]> * fix(ingestion/powerbi): fix issue with broken report lineage (datahub-project#10910) * feat(ingest/tableau): add retry on timeout (datahub-project#10995) * change generate kafka connect properties from env (datahub-project#10545) Co-authored-by: david-leifker <[email protected]> * fix(ingest): fix oracle cronjob ingestion (datahub-project#11001) Co-authored-by: david-leifker <[email protected]> * chore(ci): revert update deprecated github actions (datahub-project#10977) (datahub-project#11062) * feat(ingest/dbt-cloud): update metadata_endpoint inference (datahub-project#11041) * build: Reduce size of datahub-frontend-react image by 50-ish% (datahub-project#10878) Co-authored-by: david-leifker <[email protected]> * fix(ci): Fix lint issue in datahub_ingestion_run_summary_provider.py (datahub-project#11063) * docs(ingest): update developing-a-transformer.md (datahub-project#11019) * feat(search-test): update search tests from datahub-project#10408 (datahub-project#11056) * feat(cli): add aspects parameter to DataHubGraph.get_entity_semityped (datahub-project#11009) Co-authored-by: Harshal Sheth <[email protected]> * docs(airflow): update min version for plugin v2 (datahub-project#11065) * doc(ingestion/tableau): doc update for derived permission (datahub-project#11054) Co-authored-by: Pedro Silva <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Harshal Sheth <[email protected]> * fix(py): remove dep on types-pkg_resources (datahub-project#11076) * feat(ingest/mode): add option to exclude restricted (datahub-project#11081) * fix(ingest): set lastObserved in sdk when unset (datahub-project#11071) * doc(ingest): Update capabilities (datahub-project#11072) * chore(vulnerability): Log Injection (datahub-project#11090) * chore(vulnerability): Information exposure through a stack trace (datahub-project#11091) * chore(vulnerability): Comparison of narrow type with wide type in loop condition (datahub-project#11089) * chore(vulnerability): Insertion of sensitive information into log files (datahub-project#11088) * chore(vulnerability): Risky Cryptographic Algorithm (datahub-project#11059) * chore(vulnerability): Overly permissive regex range (datahub-project#11061) Co-authored-by: Harshal Sheth <[email protected]> * fix: update customer data (datahub-project#11075) * fix(models): fixing the datasetPartition models (datahub-project#11085) Co-authored-by: John Joyce <[email protected]> * fix(ui): Adding view, forms GraphQL query, remove showing a fallback error message on unhandled GraphQL error (datahub-project#11084) Co-authored-by: John Joyce <[email protected]> * feat(docs-site): hiding learn more from cloud page (datahub-project#11097) * fix(docs): Add correct usage of orFilters in search API docs (datahub-project#11082) Co-authored-by: Jay <[email protected]> * fix(ingest/mode): Regexp in mode name matcher didn't allow underscore (datahub-project#11098) * docs: Refactor customer stories section (datahub-project#10869) Co-authored-by: Jeff Merrick <[email protected]> * fix(release): fix full/slim suffix on tag (datahub-project#11087) * feat(config): support alternate hashing algorithm for doc id (datahub-project#10423) Co-authored-by: david-leifker <[email protected]> Co-authored-by: John Joyce <[email protected]> * fix(emitter): fix typo in get method of java kafka emitter (datahub-project#11007) * fix(ingest): use correct native data type in all SQLAlchemy sources by compiling data type using dialect (datahub-project#10898) Co-authored-by: Harshal Sheth <[email protected]> * chore: Update contributors list in PR labeler (datahub-project#11105) * feat(ingest): tweak stale entity removal messaging (datahub-project#11064) * fix(ingestion): enforce lastObserved timestamps in SystemMetadata (datahub-project#11104) * fix(ingest/powerbi): fix broken lineage between chart and dataset (datahub-project#11080) * feat(ingest/lookml): CLL support for sql set in sql_table_name attribute of lookml view (datahub-project#11069) * docs: update graphql docs on forms & structured properties (datahub-project#11100) * test(search): search openAPI v3 test (datahub-project#11049) * fix(ingest/tableau): prevent empty site content urls (datahub-project#11057) Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(entity-client): implement client batch interface (datahub-project#11106) * fix(snowflake): avoid reporting warnings/info for sys tables (datahub-project#11114) * fix(ingest): downgrade column type mapping warning to info (datahub-project#11115) * feat(api): add AuditStamp to the V3 API entity/aspect response (datahub-project#11118) * fix(ingest/redshift): replace r'\n' with '\n' to avoid token error redshift serverless… (datahub-project#11111) * fix(entiy-client): handle null entityUrn case for restli (datahub-project#11122) * fix(sql-parser): prevent bad urns from alter table lineage (datahub-project#11092) * fix(ingest/bigquery): use small batch size if use_tables_list_query_v2 is set (datahub-project#11121) * fix(graphql): add missing entities to EntityTypeMapper and EntityTypeUrnMapper (datahub-project#10366) * feat(ui): Changes to allow editable dataset name (datahub-project#10608) Co-authored-by: Jay Kadambi <[email protected]> * fix: remove saxo (datahub-project#11127) * feat(mcl-processor): Update mcl processor hooks (datahub-project#11134) * fix(openapi): fix openapi v2 endpoints & v3 documentation update * Revert "fix(openapi): fix openapi v2 endpoints & v3 documentation update" This reverts commit 573c1cb. * docs(policies): updates to policies documentation (datahub-project#11073) * fix(openapi): fix openapi v2 and v3 docs update (datahub-project#11139) * feat(auth): grant type and acr values custom oidc parameters support (datahub-project#11116) * fix(mutator): mutator hook fixes (datahub-project#11140) * feat(search): support sorting on multiple fields (datahub-project#10775) * feat(ingest): various logging improvements (datahub-project#11126) * fix(ingestion/lookml): fix for sql parsing error (datahub-project#11079) Co-authored-by: Harshal Sheth <[email protected]> * feat(docs-site) cloud page spacing and content polishes (datahub-project#11141) * feat(ui) Enable editing structured props on fields (datahub-project#11042) * feat(tests): add md5 and last computed to testResult model (datahub-project#11117) * test(openapi): openapi regression smoke tests (datahub-project#11143) * fix(airflow): fix tox tests + update docs (datahub-project#11125) * docs: add chime to adoption stories (datahub-project#11142) * fix(ingest/databricks): Updating code to work with Databricks sdk 0.30 (datahub-project#11158) * fix(kafka-setup): add missing script to image (datahub-project#11190) * fix(config): fix hash algo config (datahub-project#11191) * test(smoke-test): updates to smoke-tests (datahub-project#11152) * fix(elasticsearch): refactor idHashAlgo setting (datahub-project#11193) * chore(kafka): kafka version bump (datahub-project#11211) * readd UsageStatsWorkUnit * fix merge problems * change logo --------- Co-authored-by: Chris Collins <[email protected]> Co-authored-by: John Joyce <[email protected]> Co-authored-by: John Joyce <[email protected]> Co-authored-by: John Joyce <[email protected]> Co-authored-by: dushayntAW <[email protected]> Co-authored-by: sagar-salvi-apptware <[email protected]> Co-authored-by: Aseem Bansal <[email protected]> Co-authored-by: Kevin Chun <[email protected]> Co-authored-by: jordanjeremy <[email protected]> Co-authored-by: skrydal <[email protected]> Co-authored-by: Harshal Sheth <[email protected]> Co-authored-by: david-leifker <[email protected]> Co-authored-by: sid-acryl <[email protected]> Co-authored-by: Julien Jehannet <[email protected]> Co-authored-by: Hendrik Richert <[email protected]> Co-authored-by: Hendrik Richert <[email protected]> Co-authored-by: RyanHolstien <[email protected]> Co-authored-by: Felix Lüdin <[email protected]> Co-authored-by: Pirry <[email protected]> Co-authored-by: Hyejin Yoon <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: cburroughs <[email protected]> Co-authored-by: ksrinath <[email protected]> Co-authored-by: Mayuri Nehate <[email protected]> Co-authored-by: Kunal-kankriya <[email protected]> Co-authored-by: Shirshanka Das <[email protected]> Co-authored-by: ipolding-cais <[email protected]> Co-authored-by: Tamas Nemeth <[email protected]> Co-authored-by: Shubham Jagtap <[email protected]> Co-authored-by: haeniya <[email protected]> Co-authored-by: Yanik Häni <[email protected]> Co-authored-by: Gabe Lyons <[email protected]> Co-authored-by: Gabe Lyons <[email protected]> Co-authored-by: 808OVADOZE <[email protected]> Co-authored-by: noggi <[email protected]> Co-authored-by: Nicholas Pena <[email protected]> Co-authored-by: Jay <[email protected]> Co-authored-by: ethan-cartwright <[email protected]> Co-authored-by: Ethan Cartwright <[email protected]> Co-authored-by: Nadav Gross <[email protected]> Co-authored-by: Patrick Franco Braz <[email protected]> Co-authored-by: pie1nthesky <[email protected]> Co-authored-by: Joel Pinto Mata (KPN-DSH-DEX team) <[email protected]> Co-authored-by: Ellie O'Neil <[email protected]> Co-authored-by: Ajoy Majumdar <[email protected]> Co-authored-by: deepgarg-visa <[email protected]> Co-authored-by: Tristan Heisler <[email protected]> Co-authored-by: Andrew Sikowitz <[email protected]> Co-authored-by: Davi Arnaut <[email protected]> Co-authored-by: Pedro Silva <[email protected]> Co-authored-by: amit-apptware <[email protected]> Co-authored-by: Sam Black <[email protected]> Co-authored-by: Raj Tekal <[email protected]> Co-authored-by: Steffen Grohsschmiedt <[email protected]> Co-authored-by: jaegwon.seo <[email protected]> Co-authored-by: Renan F. Lima <[email protected]> Co-authored-by: Matt Exchange <[email protected]> Co-authored-by: Jonny Dixon <[email protected]> Co-authored-by: Pedro Silva <[email protected]> Co-authored-by: Pinaki Bhattacharjee <[email protected]> Co-authored-by: Jeff Merrick <[email protected]> Co-authored-by: skrydal <[email protected]> Co-authored-by: AndreasHegerNuritas <[email protected]> Co-authored-by: jayasimhankv <[email protected]> Co-authored-by: Jay Kadambi <[email protected]> Co-authored-by: David Leifker <[email protected]>
Checklist
Summary by CodeRabbit
New Features
TagsToTermMappertransformer that maps specific tags to glossary terms in DataHub.Documentation
Tags to Term Mappingin the dataset transformer documentation, detailing usage and configuration.Tests
TagsToTermMappertransformer.