Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.10.1

For changelog, see release notes.

Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.10.0

For changelog, see release notes.

Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.9.0

For changelog, see release notes.

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

• chore: Update RELEASE.md by @chensun in #10767
• fix(components): Add input param autorater_prompt_parameters to online_evaluation_pairwise component by @copybara-service in #10759
• feat(components): create infer preprocessor component by @copybara-service in #10765
• feat(components): create utility class for preprocessors and use it in rlhf preprocessor and infer preprocessor by @copybara-service in #10768
• chore(sdk): drop Python 3.7 for kfp sdk by @rickyxie0929 in #10750
• chore: Promote @rimolive as a backend approver by @chensun in #10780
• feat(components): use preprocessor utility methods for the upload model graph by @copybara-service in #10774
• chore: upgrade node to v18 by @HumairAK in #10794
• feat(GH workflow): migrate periodic functional tests to GH actions by @shruti2522 in #10751
• chore: Add Container Engine and Image Tag parameters to backend Makefiles by @gmfrasca in #10725
• chore: add meaningful logs to client initialization. by @HumairAK in #10801
• feat(kubernetes_platform): Update kubernetes_platform go package to include EnabledSharedMemory by @hsteude in #10703
• feat(components): Retry on batch prediction internal errors in AutoSxS by @copybara-service in #10799
• fix(components): Add staging and temp locations to prophet trainer component by @copybara-service in #10803
• fix(components): Remove unused import function_based from infer pipeline by @copybara-service in #10802
• docs(components): Bump the python version to 3.8 by @copybara-service in #10764
• chore(components): GCPC 2.14.1 Release by @copybara-service in #10813
• chore(components): Rollback GCPC 2.14.1 Release due to b/339911077 by @copybara-service in #10818
• chore(components): Add TaskError proto by @copybara-service in #10823
• chore(components): GCPC 2.14.1 Release by @copybara-service in #10821
• feat(components): Create the write_user_defined_error function by @copybara-service in #10835
• test: fix CI failure for periodic functional tests by @tmvfb in #10817
• chore(docs): Fix link by @haoxins in #7533
• chore: Update Periodic Functional Tests to run once a day by @DharmitD in #10842
• test: fix version conflict for functional tests by @tmvfb in #10837
• chore: server should skip sample load if path DNE by @DharmitD in #10833
• fix(frontend): reduce list run latency by @droctothorpe in #10797
• feat(components): Support parsing Gemini BP outputs in AutoSxS pipeline by @copybara-service in #10853
• feat(components): Use GetModel integration test to manually test write_user_defined_error function by @copybara-service in #10843
• docs: modified the ConfigMap as Env variable example in README.md with right key_name within use_config_map_as_env() block. by @vamsi-01 in #10855
• fix(sdk): Add required auth scopes to RegistryClient for GCP service accounts credentials by @PChambino in #10819
• chore(deps): bump requests from 2.31.0 to 2.32.2 in /backend/src/apiserver/visualization by @dependabot in #10858
• chore: Change command to apply manifests in the docs by @rimolive in #10864
• fix(manifests): Move metacontroller to the top in kustmization.yaml by @codablock in #10669
• test: Migrate Backend tests to a GHA workflow by @DharmitD in #10871
• feat(backend): mount EmptyDir volumes for launcher write locations by @HumairAK in #10857
• docs(components): internal by @copybara-service in #10874
• feat(components): Add role_field_name and model_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema by @copybara-service in #10875
• fix(components): add check and add log to call out the fallback to the default model checkpoint and remove the model checkpoint check condition in RLHF GCPC by @copybara-service in #10828
• test: migrate kubeflow-pipeline-e2e-test to GitHub Actions by @hbelmiro in #10887
• feat(components): Support dynamic machine type paramters in CustomTrainingJobOp by @KevinGrantLee in #10883
• fix(kubernetes_platform): fix api-generator docker mount for SELinux by @gregsheremeta in #10890
• feat(components): Add Starry Net forecasting pipeline to public preview by @copybara-service in #10888
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10906
• feat(components): Update Starry Net image tags by @copybara-service in #10920
• chore: Updated Python image to 3.8 in backend/Dockerfile by @hbelmiro in #10941
• Expose starry_net yaml to GitHub by @chensun in #10943
• chore(sdk): release KFP SDK 2.8.0 by @chensun in #10940
• chore: Upgrade Python version to 3.8 in readthedocs config by @chensun in #10945
• feat(frontend&backend): Add UI support for object store customization and prefixes by @HumairAK in #10787
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10958
• chore(components): GCPC 2.15.0 Release by @copybara-service in #10951
• docs(backend): Remove deprecated v2 compatibility mode docs by @droctothorpe in #10956
• Upgrade go version to 1.21 by @DharmitD in #10911
• chore: Add ability to mount self-signed certs to kfp by @DharmitD in #10849
• feat(components): add persistent_resource_id to v1 GCPC custom job components/utils by @copybara-service in #10954
• chore: Added @rimolive, @hbelmiro and @DharmitD as approvers and reviewers for GitHub Actions by @hbelmiro in #10963
• test: Migrate KFP SDK YAPF test to a GHA Workflow by @DharmitD in #10961
• test: Created a new GitHub action to reuse the creation of a KFP cluster by @hbelmiro in #10946
• test: kubeflow-pipeline-upgrade-test migrated to GitHub Actions by @hbelmiro in #10932
• test: Migrate Frontend tests to a GHA Workflow by @DharmitD in #10972
• test: install kfp on github periodic functional tests workflow by @tmvfb in #10859
• chore: Add an option to re-run GHA workflows by @rimolive in #10930
• chore(components): Update Starry Net description and add nan- and zero-threshold args by @copybara-service in #10986
• test: Moved KFP backend visualization tests to GitHub Actions by @hbelmiro in #10984
• test: Migrate KFP SDK Upgrade Test to GHA by @DharmitD in #10989
• test: Migrate KFP SDK Runtime Code Test to GHA by @DharmitD in #10991
• chore(components): Bump image version for Structured Data pipelines by @copybara-service in #11000
• chore(components): Fix bug due to protobuf library being upgraded by pinning protobuf version by @copybara-service in #11007
• chore(components): Add target_field_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema by @copybara-service in https://github.co...

Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.8.0

For changelog, see release notes.

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

• feat(components): Report TensorBoard metrics for preview.llm.rlhf_pipeline in real time by @copybara-service in #10595
• feat(kubernetes_platform): Update kubernetes_platform go package to include generic ephemerl volume by @abaland in #10602
• fix(metadata envoy): upgrade envoy and config from 1.12 to 1.27 by @freefood89 in #10589
• fix(components): Remove the unused resolve_data_paths from function_based by @copybara-service in #10606
• chore: Add Question issue template by @rimolive in #10557
• feat(kubernetes_platform): Update kubernetes_platform go package to include node affinities and pod (anti)affinities by @cjidboon94 in #10583
• chore(components): Add test machine spec support to preview.llm pipelines by @copybara-service in #10616
• fix(components): Ensure preview.llm.rlhf_pipeline runs if no tensorboard_id is provided by @copybara-service in #10626
• fix(components): Remove the unused functions from function_based by @copybara-service in #10613
• feat(components): Add model name preprocess component; Use publisher model if user uploaded model is non-tuned by @copybara-service in #10620
• feat(components): add task_type as a parameter to rlaif by @copybara-service in #10607
• feat(components): AutoSxS GA pending release by @copybara-service in #10628
• fix(components): Update service account comment by @copybara-service in #10636
• chore(components): GCPC 2.12.0 Release by @copybara-service in #10635
• fix(samples): Fix the loop_output.py example to handle the new parallel loop type requirement by @Tomcli in #10637
• feat(components): Added support for text-bison@002 to preview.llm.rlhf_pipeline by @copybara-service in #10641
• chore(components): GCPC 2.13.0 Release by @copybara-service in #10644
• fix(components): Make AutoSxS autorater_prompt_parameters required by @copybara-service in #10621
• docs(components): internal by @copybara-service in #10649
• chore(components): Update kserve component to v0.12.0 by @Tomcli in #10652
• fix(backend): Update backend common code and integration tests with updated API Service Params by @gmfrasca in #10640
• fix(Backend + SDK): Add missing optional field to SecretAsVolume and … by @revit13 in #10550
• chore(manifests): refactor manifests for kustomize5 compatibility. Part of #10053 by @rawc0der in #10087
• fix(components): Fix model eval import error in text generation/classification eval pipeline by @copybara-service in #10655
• chore(components): GCPC 2.13.1 Release by @copybara-service in #10666
• chore(deps): bump follow-redirects from 1.6.1 to 1.15.6 in /frontend by @dependabot in #10575
• chore(deps): bump pillow from 10.0.1 to 10.3.0 in /backend/src/apiserver/visualization by @dependabot in #10658
• fix(components): Remove the unused generate_default_instruction and resolve_upload_location from function_based by @copybara-service in #10638
• chore(deps): bump express from 4.18.2 to 4.19.2 in /frontend by @dependabot in #10639
• chore: Update argo images to 3.4.16 by @chensun in #10618
• feat(components): Use larger base reward model when tuning text and chat variants of bison@001 with the preview.llm.rlhf_pipeline by @copybara-service in #10663
• chore(deps): bump follow-redirects from 1.5.10 to 1.15.6 in /frontend/server by @dependabot in #10574
• chore(sample): add note about secret needing to be pre-created by @gregsheremeta in #10659
• Chore(components): Clean up old ibm components by @Tomcli in #10680
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10683
• feat(sdk+backend): Add support for generic ephemeral volume by @abaland in #10605
• feat(components): Use larger base reward model when tuning t5-xxl with the preview.llm.rlhf_pipeline by @copybara-service in #10665
• chore(backend): Promote @rimolive as the backend reviewer by @Tomcli in #10689
• chore(sdk): release kfp-kubernetes 1.2.0 by @connor-mccarthy in #10692
• chore(sdk): make kfp-kubernetes release instructions public by @connor-mccarthy in #10693
• chore(kfp-kubernetes): change type of affinity weight to int32 by @cjidboon94 in #10671
• fix(components): Fix image version parameter in rl pipelines by @copybara-service in #10698
• docs(components): internal by @copybara-service in #10694
• feat(backend): Merge kfp-tekton backend code by @rimolive in #10678
• feat(backend): Upgrade argo to v3.4.16 by @gmfrasca in #10568
• feat(backend): add namespace & prefix scoped credentials to kfp-launcher config for object store paths by @HumairAK in #10625
• chore(deps): bump sqlparse from 0.4.4 to 0.5.0 in /backend/src/apiserver/visualization by @dependabot in #10700
• feat(components): add resolve_reference_model_metadata to rlhf_preprocessor component by @copybara-service in #10612
• feat(components): Move AutoSxS pipeline to v1 directory by @copybara-service in #10701
• feat(components): add resolve_machine_spec and resolve_refined_image_uri to rlhf_preprocessor component by @copybara-service in #10608
• fix(components): remove default prediction column names in evaluation classification component to fix incorrect column names for bigquery data source by @copybara-service in #10708
• feat(components): Introduce placeholders: SERVICE_ACCOUNT_PLACEHOLDER, NETWORK_PLACEHOLDER, PERSISTENT_RESOURCE_ID_PLACEHOLDER and ENCYRPTION_SPEC_KMS_KEY_NAME_PLACEHOLDER. In addition, use PERSISTENT_RESOURCE_ID_PLACEHOLDER as the default value of persistent_resource_id for CustomTrainingJobOp and create_custom_training_job_op_from_component. With this change, custom job created without explicitly setting persistent_resource_id will inherit job level persistent_resource_id, if Persistent Resource is set as job level runtime by @copybara-service in #10650
• chore: Add kfp-tekton integration tests and manifests by @rimolive in #10702
• No public description by @copybara-service in #10726
• chore(components): Drop GCPC Python 3.7 by @copybara-service in #10730
• feat(components): use rlhf_preprocessor to replace the current value_exists call in rlhf by @copybara-service in #10584
• chore(components): Drop support for Python 3.7 in GCPC by @copybara-service in #10735
• feat(components): internal by @copybara-service in #10707
• feat(components): Expand regions supported by preview.llm.rlhf_pipeline by @copybara-service in #10710
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10749
• chore(components): Change the warning for Python 3.8 by @copybara-service in #10755
• feat(components): migrate function_based resolve_num_microbatches to rlhf_preprocessor component by @copybara-service in #10604
• chore(components): GCPC 2.14.0 Release by @copybara-service in #10754
• feat(components): migrate function_based convert_to_delimited_string to rlhf_preprocessor component by @copybara-service in https://github.com/kube...

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

• feat(components): Bump image tag used by preview.llm pipelines by @copybara-service in #10295
• feat(components): Add num_microbatches to _implementation.llm training components by @copybara-service in #10248
• chore(sdk): remove cleanup param in local init #localexecution by @connor-mccarthy in #10293
• feat(sdk): add local execution skeleton #localexecution by @connor-mccarthy in #10292
• fix(components): Use llama-2-7b for the base reward model when tuning llama-2-13 with the preview.llm.rlhf_pipeline by @copybara-service in #10249
• feat(sdk): add local task dispatcher validation and ExecutorInput construction #localexecution by @connor-mccarthy in #10298
• test(sdk): install kfp-pipeline-spec from source in kfp sdk tests by @connor-mccarthy in #10300
• fix(sdk): use kfp.dsl.types to replace kfp.components.types Fixes #10282 by @hsinhoyeh in #10283
• feat(components): Enable text generation pipeline to generate row based metrics by @copybara-service in #10296
• test(sdk): add placeholder_utils_test.py by @connor-mccarthy in #10301
• feat(component): Migrate AutoSxS pipeline to preview and move related files to _implementation/llm directory to help Model Eval team use side by side metrics as part of their pipeline by @copybara-service in #10294
• feat(components): Release new LLM Eval image version 0.5 by @copybara-service in #10313
• chore(components): release GCPC 2.7.0 by @copybara-service in #10303
• fix(components): Add autosxs_pipeline to the all variable for the preview/model_evaluation directory by @copybara-service in #10317
• chore(components): release GCPC 2.8.0 by @copybara-service in #10316
• feat(components): Add better docstrings for AutoSxS by @copybara-service in #10320
• test(components): INTERNAL by @copybara-service in #10314
• feat(sdk): add subprocess task handler #localexecution by @connor-mccarthy in #10302
• fix(components): Use large_model_reference as model_reference_name when uploading models from preview.llm.rlhf_pipeline instead of hardcoding value as text-bison@001 by @copybara-service in #10321
• test(sdk): add kfp-kubernetes execution tests by @connor-mccarthy in #10304
• fix(components): Resolve unique model display name on each preview.llm.rlhf_pipeline run instead of reusing cached result by @copybara-service in #10322
• fix(components): Upload the tuned adapter to Model Registry instead of model checkpoint from preview.llm.rlhf_pipeline by @copybara-service in #10323
• feat(sdk): add local execution output collection #localexecution by @connor-mccarthy in #10325
• chore(sdk): fix use of invalid escape sequence in tests by @connor-mccarthy in #10310
• feat(sdk): add local execution logging #localexecution by @connor-mccarthy in #10326
• chore(components): Fix argument description by @copybara-service in #10327
• feat(backend): preserve querystring in pipeline root (fixes #10318) by @TobiasGoerke in #10319
• feat(components): change output format to allow possible post eval by @copybara-service in #10281
• fix(frontend): Add disableParsingRawHTML option for markdown-to-jsx component by @zijianjoy in #10315
• feat(sdk): add DockerRunner #localexecution by @connor-mccarthy in #10328
• feat(sdk): add special dsl.OutputPath read logic #localexecution by @connor-mccarthy in #10334
• chore(sdk): write local execution logs to stdout #localexecution by @connor-mccarthy in #10330
• feat(sdk): support local Container Component execution #localexecution by @connor-mccarthy in #10333
• feat(components): Update RLHF env vars to handle empty string by @copybara-service in #10331
• feat(components): Output errors as a separate table from Arbiter by @copybara-service in #10329
• docs(components): Fix AutoSxS docstring formatting by @copybara-service in #10341
• chore(components): add json_escape placeholder util by @copybara-service in #10351
• fix(sdk): fix presentation of strings in local execution #localexecution by @connor-mccarthy in #10353
• fix(sdk): remove redundant newline character in local DockerRunner logs #localexecution by @connor-mccarthy in #10354
• test(sdk): improve KFP SDK local runner test safety #localexecution by @connor-mccarthy in #10336
• feat(backend): Enable logging for KFP components by @DharmitD in #10288
• feat(sdk): remove local execution feature flag #localexecution by @connor-mccarthy in #10355
• fix(sdk): permit empty local execution outputs #localexecution by @connor-mccarthy in #10338
• feat(sdk): support Concat and IfPresent placeholder in local container component execution #localexecution by @connor-mccarthy in #10348
• test(sdk): add test for local execution of None default parameter #localexecution by @connor-mccarthy in #10339
• chore(sdk): release KFP SDK 2.5.0 by @connor-mccarthy in #10364
• chore(sdk): depend on protobuf 4 in kfp-pipeline-spec by @connor-mccarthy in #10305
• docs(components): Document AutoML Tables util functions by @copybara-service in #10359
• feat(components): Implement the feature store grounding pipeline by @copybara-service in #10332
• feat(components): update eval pipeline documentation to clarify the required pipeline parameters by @copybara-service in #10366
• docs(components): update BigqueryQueryJobOp docs on container args limit by @copybara-service in #10380
• chore(components): GCPC 2.8.1 release by @copybara-service in #10381
• feat(kubernetes_platform): Update kubernetes_platform go package to include pod labels and annotations by @Tomcli in #10357
• chore(sdk): depend on protobuf 4 in kfp and kfp-kubernetes by @connor-mccarthy in #10307
• feat(components): Implement new output format of inference component by @copybara-service in #10375
• chore(components): rollback GCPC 2.8.1 release by @copybara-service in #10385
• chore(sdk): release KFP SDK 2.6.0 by @connor-mccarthy in #10386
• chore(components): Add v1.model.ModelGetOp components by @copybara-service in #10226
• chore(sdk): release kfp-kubernetes 1.1.0 by @connor-mccarthy in #10387
• chore(sdk): update kfp-kubernetes docs versions and release scripts by @connor-mccarthy in #10388
• docs(sdk): fix kfp-kubernetes docs build error by @connor-mccarthy in #10389
• chore(components): Sync AutoML components by @copybara-service in #10372
• fix(components): Update base image for KFP lightweight component for VPC SC compliance by @copybara-service in #10374
• chore: Fix metrics visualization v2 sample by @rimolive in #10399
• docs(sdk): add kfp.local to reference docs #localexecution by @connor-mccarthy in #10395
• chore(components): migrate GCPC to protobuf 4; require KFP>=2.6.0 by @copybara-service in #10401
• feat(components): Support scheduling and labels in utils.bui...

Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.7.0

For changelog, see release notes.

Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.6.0

For changelog, see release notes.

Release of the KFP SDK only.

To install the KFP SDK:

pip install kfp==2.5.0

For changelog, see release notes.