To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

• feat(components): Report TensorBoard metrics for preview.llm.rlhf_pipeline in real time by @copybara-service in #10595
• feat(kubernetes_platform): Update kubernetes_platform go package to include generic ephemerl volume by @abaland in #10602
• fix(metadata envoy): upgrade envoy and config from 1.12 to 1.27 by @freefood89 in #10589
• fix(components): Remove the unused resolve_data_paths from function_based by @copybara-service in #10606
• chore: Add Question issue template by @rimolive in #10557
• feat(kubernetes_platform): Update kubernetes_platform go package to include node affinities and pod (anti)affinities by @cjidboon94 in #10583
• chore(components): Add test machine spec support to preview.llm pipelines by @copybara-service in #10616
• fix(components): Ensure preview.llm.rlhf_pipeline runs if no tensorboard_id is provided by @copybara-service in #10626
• fix(components): Remove the unused functions from function_based by @copybara-service in #10613
• feat(components): Add model name preprocess component; Use publisher model if user uploaded model is non-tuned by @copybara-service in #10620
• feat(components): add task_type as a parameter to rlaif by @copybara-service in #10607
• feat(components): AutoSxS GA pending release by @copybara-service in #10628
• fix(components): Update service account comment by @copybara-service in #10636
• chore(components): GCPC 2.12.0 Release by @copybara-service in #10635
• fix(samples): Fix the loop_output.py example to handle the new parallel loop type requirement by @Tomcli in #10637
• feat(components): Added support for text-bison@002 to preview.llm.rlhf_pipeline by @copybara-service in #10641
• chore(components): GCPC 2.13.0 Release by @copybara-service in #10644
• fix(components): Make AutoSxS autorater_prompt_parameters required by @copybara-service in #10621
• docs(components): internal by @copybara-service in #10649
• chore(components): Update kserve component to v0.12.0 by @Tomcli in #10652
• fix(backend): Update backend common code and integration tests with updated API Service Params by @gmfrasca in #10640
• fix(Backend + SDK): Add missing optional field to SecretAsVolume and … by @revit13 in #10550
• chore(manifests): refactor manifests for kustomize5 compatibility. Part of #10053 by @rawc0der in #10087
• fix(components): Fix model eval import error in text generation/classification eval pipeline by @copybara-service in #10655
• chore(components): GCPC 2.13.1 Release by @copybara-service in #10666
• chore(deps): bump follow-redirects from 1.6.1 to 1.15.6 in /frontend by @dependabot in #10575
• chore(deps): bump pillow from 10.0.1 to 10.3.0 in /backend/src/apiserver/visualization by @dependabot in #10658
• fix(components): Remove the unused generate_default_instruction and resolve_upload_location from function_based by @copybara-service in #10638
• chore(deps): bump express from 4.18.2 to 4.19.2 in /frontend by @dependabot in #10639
• chore: Update argo images to 3.4.16 by @chensun in #10618
• feat(components): Use larger base reward model when tuning text and chat variants of bison@001 with the preview.llm.rlhf_pipeline by @copybara-service in #10663
• chore(deps): bump follow-redirects from 1.5.10 to 1.15.6 in /frontend/server by @dependabot in #10574
• chore(sample): add note about secret needing to be pre-created by @gregsheremeta in #10659
• Chore(components): Clean up old ibm components by @Tomcli in #10680
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10683
• feat(sdk+backend): Add support for generic ephemeral volume by @abaland in #10605
• feat(components): Use larger base reward model when tuning t5-xxl with the preview.llm.rlhf_pipeline by @copybara-service in #10665
• chore(backend): Promote @rimolive as the backend reviewer by @Tomcli in #10689
• chore(sdk): release kfp-kubernetes 1.2.0 by @connor-mccarthy in #10692
• chore(sdk): make kfp-kubernetes release instructions public by @connor-mccarthy in #10693
• chore(kfp-kubernetes): change type of affinity weight to int32 by @cjidboon94 in #10671
• fix(components): Fix image version parameter in rl pipelines by @copybara-service in #10698
• docs(components): internal by @copybara-service in #10694
• feat(backend): Merge kfp-tekton backend code by @rimolive in #10678
• feat(backend): Upgrade argo to v3.4.16 by @gmfrasca in #10568
• feat(backend): add namespace & prefix scoped credentials to kfp-launcher config for object store paths by @HumairAK in #10625
• chore(deps): bump sqlparse from 0.4.4 to 0.5.0 in /backend/src/apiserver/visualization by @dependabot in #10700
• feat(components): add resolve_reference_model_metadata to rlhf_preprocessor component by @copybara-service in #10612
• feat(components): Move AutoSxS pipeline to v1 directory by @copybara-service in #10701
• feat(components): add resolve_machine_spec and resolve_refined_image_uri to rlhf_preprocessor component by @copybara-service in #10608
• fix(components): remove default prediction column names in evaluation classification component to fix incorrect column names for bigquery data source by @copybara-service in #10708
• feat(components): Introduce placeholders: SERVICE_ACCOUNT_PLACEHOLDER, NETWORK_PLACEHOLDER, PERSISTENT_RESOURCE_ID_PLACEHOLDER and ENCYRPTION_SPEC_KMS_KEY_NAME_PLACEHOLDER. In addition, use PERSISTENT_RESOURCE_ID_PLACEHOLDER as the default value of persistent_resource_id for CustomTrainingJobOp and create_custom_training_job_op_from_component. With this change, custom job created without explicitly setting persistent_resource_id will inherit job level persistent_resource_id, if Persistent Resource is set as job level runtime by @copybara-service in #10650
• chore: Add kfp-tekton integration tests and manifests by @rimolive in #10702
• No public description by @copybara-service in #10726
• chore(components): Drop GCPC Python 3.7 by @copybara-service in #10730
• feat(components): use rlhf_preprocessor to replace the current value_exists call in rlhf by @copybara-service in #10584
• chore(components): Drop support for Python 3.7 in GCPC by @copybara-service in #10735
• feat(components): internal by @copybara-service in #10707
• feat(components): Expand regions supported by preview.llm.rlhf_pipeline by @copybara-service in #10710
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10749
• chore(components): Change the warning for Python 3.8 by @copybara-service in #10755
• feat(components): migrate function_based resolve_num_microbatches to rlhf_preprocessor component by @copybara-service in #10604
• chore(components): GCPC 2.14.0 Release by @copybara-service in #10754
• feat(components): migrate function_based convert_to_delimited_string to rlhf_preprocessor component by @copybara-service in #10600
• feat(components): Move ModelImportEvaluationOp component to preview namespace by @copybara-service in #10712
• chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10763
• feat(components): migrate function_based resolve_regional_endpoint to rlhf_preprocessor component by @copybara-service in #10603
• chore(backend): update kfp launcher and driver images by @chensun in #10733

New Contributors

• @freefood89 made their first contribution in #10589
• @cjidboon94 made their first contribution in #10583
• @gmfrasca made their first contribution in #10640
• @rawc0der made their first contribution in #10087
• @gregsheremeta made their first contribution in #10659

Full Changelog: 2.1.0...2.2.0