Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] Execute KFP pipelines locally: Unable to use DockerRunner() #11418

Open
akinwilson opened this issue Nov 28, 2024 · 0 comments
Open

[bug] Execute KFP pipelines locally: Unable to use DockerRunner() #11418

akinwilson opened this issue Nov 28, 2024 · 0 comments
Labels

Comments

@akinwilson
Copy link

akinwilson commented Nov 28, 2024

Environment

NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.1 LTS (Noble Numbat)"
VERSION_CODENAME=noble

KFP version and docker version

- kfp==2.10.1
- kfp-pipeline-spec==0.5.0
- kfp-server-api==2.3.0
- Docker version 27.3.1, build ce12230

All I am trying to do to is test out the kubeflow pipeline feature locally without deploying all the infrastructure. As far as I am aware, all I need is docker to be installed and I should be good to go . I am following this guide: and when it comes to running the code:

from kfp import local
from kfp import dsl

local.init(runner=local.DockerRunner())

@dsl.component
def add(a: int, b: int) -> int:
    return a + b

# run a single component
task = add(a=1, b=2)
assert task.output == 3

# or run it in a pipeline
@dsl.pipeline
def math_pipeline(x: int, y: int, z: int) -> int:
    t1 = add(a=x, b=y)
    t2 = add(a=t1.output, b=z)
    return t2.output

pipeline_task = math_pipeline(x=1, y=2, z=3)
assert pipeline_task.output == 6

I get the following error:

ImportError: cannot import name 'local' from partially initialized module 'kfp' (most likely due to a circular import) (/home/ola/Code/mlops/apps/kfp.py)

Can anyone let me know what it is that I am doing that is wrong? I thought I could use my local docker engine to simulate these pipelines without the need of a KF cluster deployed.

What am I doing wrong?

@akinwilson akinwilson changed the title [bug] <Execute KFP pipelines locally: Unable to use DockerRunner() > [bug] Execute KFP pipelines locally: Unable to use DockerRunner() Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant