Description
Environment
-
How did you deploy Kubeflow Pipelines (KFP)?
local deployment
Kubeflow manifest 1.9.1
k8s version:1.30.3
deployed multi-user version kuebflow with the kustomization file in the example folder.
all images are replaced with images from local storage
GKE disabled -
KFP version:
kubeflow pipeline 2.3.0
Steps to reproduce
- checked the kustomization file to see which version of pipeline i deployed
# Kubeflow Pipelines
- ../apps/pipeline/upstream/env/cert-manager/platform-agnostic-multi-user
-
click pipeline-experiment in the kubeflow UI, then click an experiment, and click compare,it says failed to get associated experiment
-
Then with F12, in pipeline-experiment page, when click any experiment or try compare,the result as follow
it says"List experiments failed: Failed to authorize with API: Invalid input error: An experiment cannot have an empty namespace in multi-user mode"
which is refer to the content in the file/backend/src/apiserver/server/experiment_server.go
https://github.com/kubeflow/pipelines/blob/58b2d8d721378fca4f35079f2aaaed92ad6bd36f/backend/src/apiserver/server/experiment_server.go#L342C1-L343C1
4.check if experiment have namespace
All the kubeflow pods are running fine
Also,I want to know if anyone else encountered the same problem or only me.
kubeflow-user-example-com ctest-0 2/2 Running 0 14d
kubeflow-user-example-com jupyter-0 2/2 Running 0 42h
kubeflow-user-example-com ml-pipeline-ui-artifact-d6bb58c54-grmxj 2/2 Running 2 (16h ago) 20d
kubeflow-user-example-com ml-pipeline-visualizationserver-745dbcb7fb-rfqpz 2/2 Running 0 20d
kubeflow-user-example-com mnist-pipeline-tsdhj-system-container-impl-1440812143 1/2 ImagePullBackOff 0 7d23h
kubeflow-user-example-com pvcviewer-codeserver-test-69b55fb6f7-4khh4 2/2 Running 0 14d
kubeflow-user-example-com pvcviewer-test00-workspace-74cb74f89f-d4s8t 2/2 Running 0 17d
kubeflow-user-example-com pvcviewer-workspace-8485f49f4f-wkmm7 2/2 Running 0 2d23h
kubeflow-user-example-com tensortst-8dcd5dcdb-h77qw 2/2 Running 0 17d
kubeflow-user-example-com test10-0 2/2 Running 2 (3d ago) 15d
kubeflow admission-webhook-deployment-5c5959bd95-rnmr9 1/1 Running 0 20d
kubeflow cache-server-784dff7cfd-768b7 2/2 Running 0 20d
kubeflow centraldashboard-56ff7d88c5-6c8dn 2/2 Running 0 9d
kubeflow jupyter-web-app-deployment-648886f787-4zxx5 2/2 Running 0 13d
kubeflow katib-controller-f8df57f77-cqnxp 1/1 Running 0 20d
kubeflow katib-db-manager-5dfb479d86-c9z22 1/1 Running 0 20d
kubeflow katib-mysql-5c54f57f6b-4m2hj 1/1 Running 0 20d
kubeflow katib-ui-6c587c9466-fnzhj 2/2 Running 0 20d
kubeflow kserve-controller-manager-854b47d55d-wmvg4 2/2 Running 0 20d
kubeflow kserve-models-web-app-d7c86d76d-wqnfm 2/2 Running 0 20d
kubeflow kubeflow-pipelines-profile-controller-7f78669c58-4r5pj 1/1 Running 0 20d
kubeflow metacontroller-0 1/1 Running 0 20d
kubeflow metadata-envoy-deployment-d4b98db88-qjk25 1/1 Running 0 20d
kubeflow metadata-grpc-deployment-6dd4c5fcb6-gvllw 2/2 Running 2 (20d ago) 20d
kubeflow metadata-writer-c98794db9-kh52q 2/2 Running 0 20d
kubeflow minio-586bbdcc4-fcszh 2/2 Running 0 8d
kubeflow ml-pipeline-647947d6d4-5l7tv 2/2 Running 0 20d
kubeflow ml-pipeline-persistenceagent-f4466586-q8wz4 2/2 Running 0 20d
kubeflow ml-pipeline-scheduledworkflow-55965d9f58-l4xxh 2/2 Running 0 20d
kubeflow ml-pipeline-ui-558979f6d4-2fj4j 2/2 Running 0 18d
kubeflow ml-pipeline-viewer-crd-7db4586d47-z27fv 2/2 Running 1 (20d ago) 20d
kubeflow ml-pipeline-visualizationserver-b687cd666-m6zdf 2/2 Running 0 20d
kubeflow mysql-675c9859d8-jklq4 2/2 Running 0 20d
kubeflow notebook-controller-deployment-546c7d4cb6-8qp5b 2/2 Running 2 (20d ago) 20d
kubeflow profiles-deployment-5cb7c65899-k77f5 3/3 Running 1 (20d ago) 20d
kubeflow pvcviewer-controller-manager-d64c9564f-9vnv4 3/3 Running 0 20d
kubeflow tensorboard-controller-deployment-5c9c98bcdf-58gfk 3/3 Running 2 (17d ago) 17d
kubeflow tensorboards-web-app-deployment-6bbd97c7d4-22cjr 2/2 Running 0 20d
kubeflow training-operator-54f754bfcd-sq7k4 1/1 Running 0 20d
kubeflow volumes-web-app-deployment-64cdf449dd-mfhk8 2/2 Running 0 17d
kubeflow workflow-controller-986648d9b-v4mzf 2/2 Running 1 (13d ago) 13d
lsq-pace ml-pipeline-ui-artifact-d6bb58c54-bsggh 2/2 Running 0 17d
lsq-pace ml-pipeline-visualizationserver-745dbcb7fb-6ssxg 2/2 Running 0 17d
oauth2-proxy oauth2-proxy-57d8b58449-6vfvz 1/1 Running 0 20d
oauth2-proxy oauth2-proxy-57d8b58449-wjbk4 1/1 Running 0 20d
tigera-operator tigera-operator-76ff79f7fd-4bd7f 1/1 Running 0 25d
Expected result
- the experiment of the runs shows correctly in the compare run page
- the experiment information fetched correctly in the compare run page and the pipeline-experiment-'my experiment' page
Materials and Reference
Impacted by this bug? Give it a 👍.