Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[backend] Components with optional artifacts not caching after migrating to kfp>2.0 #11410

Open
jortizta opened this issue Nov 26, 2024 · 0 comments

Comments

@jortizta
Copy link

After migrating a collection of custom components and pipelines running in Vertex AI from KFP 1.8 to KFP 2.3 and 2.8, the components which have optional artifacts are no longer cached.

Environment

  • KFP version: 2.3, 2.8
  • KFP SDK version: 2.3, 2.8
  • Platform: Vertex AI

Steps to reproduce

Compile, and run a custom component with an optional artifact, for example:

name: Train XGBoost
description: Train an XGBoost
inputs:
  - {name: data_artifact, type: Dataset, optional: True, description: 'Dataset used for training'}
  - {name: data_path, type: String, description: 'Path to parquet file used for training'}
  - {name: target_variable, type: String, description: ''}
  
outputs:
  - {name: model_prod_artifact, type: Model}
  
implementation:
  container:
    image: gcr.io/PROJECT_ID/component_default@DIGEST
    args:
    - --executor_input
    - executorInput: null
    - --function_to_execute
    - train_xgb
    command: 
    - python3
    - -m 
    - kfp.dsl.executor_main
    - --component_module_path
    - ./train_xgb.py

Expected result

Having optional artifacts in the component definition should not prevent component caching from working. That was the behavior in KFP 1.8


Impacted by this bug? Give it a 👍.

@jortizta jortizta changed the title [backend] Components with optional artifacts not caching after migrating to kfp>=2.0 [backend] Components with optional artifacts not caching after migrating to kfp>2.0 Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant