Add missing attribute "name" and "group" for Asset and "group" for AssetAlias in serialization, api and methods #43774
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
boring-cyborg
bot
added
area:serialization
area:webserver
Member
Author
TODOExtend the following test case to include more asset attributes
|
Lee-W
force-pushed
the
fix-asset-name-uri-handling
branch
from
32394ac to
4726cc2
Compare
Lee-W
force-pushed
the
fix-asset-name-uri-handling
branch
from
4726cc2 to
0b6a309
Compare
Lee-W
force-pushed
the
fix-asset-name-uri-handling
branch
6 times, most recently
from
a57ed8b to
9f9366d
Compare
Member
|
Makes sense to me. Will take another look when this is ready for review. |
Lee-W
force-pushed
the
fix-asset-name-uri-handling
branch
4 times, most recently
from
5473c2f to
678640e
Compare
Lee-W
force-pushed
the
fix-asset-name-uri-handling
branch
4 times, most recently
from
a219f5b to
cb65d48
Compare
Lee-W
force-pushed
the
fix-asset-name-uri-handling
branch
5 times, most recently
from
a7f2353 to
d52ad63
Compare
…warning to cover extra edge case
…p fields (also AssetAlias group field)
…d group to asset alias schema
…ciation as asset_model
…reate asset event
…instead of (uri, obj)
… to remove duplicate code
This reverts commit e812b8a.
Lee-W
force-pushed
the
fix-asset-name-uri-handling
branch
from
ab92813 to
981011e
Compare
uranusjr
approved these changes
Dec 2, 2024
Member
uranusjr
left a comment
uranusjr
left a comment
There was a problem hiding this comment.
This makes sense. I wonder if we can manage the DagDependency values better… but maybe it’s not in scope for this PR.
Member
Author
probably need to discuss with brent maybe 🤔 |
got686-yandex
pushed a commit
to got686-yandex/airflow
that referenced
this pull request
Jan 30, 2025
…setAlias in serialization, api and methods (apache#43774) * test(tests/www/views/test_views_grid): extend Asset test cases to include both uri and name * test(utils/test_json): extend Asset test cases to include both uri and name * test(timetables/test_assets_timetable): extend Asset test cases to include both uri and name * test(listeners/test_asset_listener): extend Asset test cases to include both uri and name * test(jobs/test_scheduler_job): extend Asset test cases to include both uri and name * test(providers/openlineage): extend Asset test cases to include both uri and name * test(decorators/test_python): extend Asset test cases to include both uri and name * test(models/test_dag): extend asset test cases to cover name, uri, group * test(api_connexsion/schemas/dag_run): extend asset test cases to cover name, uri, group * test(serialization/serialized_objects): extend asset test cases to cover name, uri, group and asset alias test cases to cover name and group * test(serialization/dag_serialization): extend asset test cases to cover name, uri, group * test(models/dag): extend asset test cases to cover name, uri, group * test(serialization/serde): extend asset test cases to cover name, uri, group * test(api_connexion/schemas/asset): extend asset test cases to cover name, uri, group * test(api_connexion/schemas/asset): extend asset alias test cases to cover name, group * test(api_connexsion/schemas/dag): extend asset test cases to cover name, uri, group * test(api_connexsion/schemas/dag_run): extend asset test cases to cover name, uri, group * test(dags/test_assets): extend asset test cases to cover name, uri, group * test(dags/test_only_empty_tasks): extend asset test cases to cover name, uri, group * test(api_fastapi): extend asset test cases to cover name, uri, group * test(assets/manager): extend asset test cases to cover name, uri, group * test(task_sdk/assets): extend asset test cases to cover name, uri, group * test(api_connexion/endpoints/asset): extend asset test cases to cover name, uri, group * test: add missing session * test(www/views/asset): extend asset test cases to cover name, uri, group * test(models/seraialized_dag): extend asset test cases to cover name, uri, group * test(lineage/hook): extend asset test cases to cover name, uri, group * test(io/path): extend asset test cases to cover name, uri, group * test(jobs): enhance test_activate_referenced_assets_with_no_existing_warning to cover extra edge case * fix(serialization): serialize both name, uri and group for Asset * fix(assets): extend Asset as_expression methods to include name, group fields (also AssetAlias group field) * fix(serialization/serialized_objects): fix missing AssetAlias.group serialization * fix(serialization): change dependency_id to use name instead of uri * feat(api_connexion/schemas/asset): add name, group to asset schema and group to asset alias schema * feat(assets/manager): filter asset by name, uri, group instead of uri only * style(assets/manager): rename argument asset in _add_asset_alias_association as asset_model * fix(asset): use name to evalute instead of uri * fix(api_connexion/endpoints/asset): fix how asset event is fetch in create asset event * fix(api_fastapi/asset): fix how asset event is fetch in create asset event * fix(lineage/hook): extend asset realted methods to include name and group * fix(task_sdk/asset): change iter_assets to return ((name, uri), obj) instead of (uri, obj) * fix(fastapi/asset): add missing group column to asset alias schema * build: build autogen ts files * feat(lineage/hook): make create_asset keyword only * docs(newsfragments): add 43774.significant.rst * refactor(task_sdk/asset): add from_asset_alias to AssetAliasCondition to remove duplicate code * refactor(task_sdk/asset): add AssetUniqueKey.from_asset to reduce duplicate code * Revert "fix(asset): use name to evalute instead of uri" This reverts commit e812b8a.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why
asset attribute name and group are not respected in many aspect after introuduced
What
dependency_idto use asset name instead of uriiter_assetsto return((name, uri), obj)instead of(uri, obj)evaluateinstead of "uri"Close: #43958
Extend the following test case to include more asset attributes (i.e., name, group)
[ ] tests/io/test_wrapper.pyit looks like it makes more sense to keep it as it[ ] tests/models/test_taskinstance.pydefer to Deprecate accessing inlet and outlet events through string #43959Rest API stuff will be handled at #44412
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.