Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

containerd + nerdctl support #1533

Open
1 of 2 tasks
secana opened this issue Aug 2, 2024 · 11 comments
Open
1 of 2 tasks

containerd + nerdctl support #1533

secana opened this issue Aug 2, 2024 · 11 comments
Labels
feature-request

Comments

@secana
Copy link

secana commented Aug 2, 2024

Checklist

Describe your request

Feature request: Support containerd + nerdctl additionally to dockerd (moby) and podman.

Currently, if cross-rs is run on a machine with containerd + nerdctl the following error pops up:

> cross build --target aarch64-unknown-linux-gnu
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.

It would be great, if other container runtimes would be detected and used. If this is not much work and can be done by someone unfamiliar with the code of the cross project, feel free to point me to the right direction.

Describe why this would be a good inclusion for cross

With the commercialisation of docker desktop, many developers switched to other solutions like Rancher Desktop. Per default containerd + nerdctl is used instead of docker to run containers.

The whole Kubernetes ecosystems seems to move rapidly into the direction of containerd, so I think it would be great if cross-rs could use that, too.
@Emilgardis
Copy link
Member

We should already support this, can you run with -v to show how nerdctl was invoked and what failed.

We detect what is used here and we query some more information from the container engine. I think that's what is going wrong

@secana
Copy link
Author

secana commented Aug 5, 2024 •
edited
Loading

Sure, here is the output:

cross build --target aarch64-unknown-linux-gnu -v
+ cargo metadata --format-version 1 --filter-platform aarch64-unknown-linux-gnu
+ rustc --print sysroot
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /Users/X/.rd/bin/docker
+ /Users/X/.rd/bin/docker run --userns host -e 'PKG_CONFIG_ALLOW_CROSS=1' -e 'XARGO_HOME=/xargo' -e 'CARGO_HOME=/cargo' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e TERM -e 'USER=X' --rm --user 1707927531:1282186731 -v /Users/X/.xargo:/xargo:z -v /Users/X/.cargo:/cargo:z -v /cargo/bin -v /Users/X/repos/gdcli:/project:z -v /Users/X/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/rust:z,ro -v /Users/X/repos/gdcli/target:/target:z -w /project -i -t ghcr.io/cross-rs/aarch64-unknown-linux-gnu:0.2.5 sh -c 'PATH=$PATH:/rust/bin cargo build --target aarch64-unknown-linux-gnu -v'
docker: error during connect: Head "http://%2Fvar%2Frun%2Fdocker.sock/_ping": EOF.
See 'docker run --help'.
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu

Some more information about the system:

> nerdctl -v
nerdctl version 1.7.6

> sw_vers
ProductName:		macOS
ProductVersion:		14.6
BuildVersion:		23G80

> cross -V
cross 0.2.5

I use Rancher Desktop 1.14.1 to manage containerd and nerdctl.

@Emilgardis
Copy link
Member

Can you try updating cross to the newest master? cargo install cross --git https://github.com/cross-rs/cross

@secana
Copy link
Author

secana commented Aug 6, 2024

Hi! I updated to the latest cross version from git.

> cross -V
cross 0.2.5 (7b79041 2024-07-11)

> cross build --target aarch64-unknown-linux-gnu
Error:
   0: could not get os and arch
   1: `docker version -f '{{ .Server.Os }},,,{{ .Server.Arch }}'` failed with exit status: 1

Location:
   src/errors.rs:147

Error:
   0: could not get os and arch
   1: `docker version -f '{{ .Client.Os }},,,{{ .Client.Arch }}'` failed with exit status: 1

Stderr:
   error during connect: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/version": EOF

@Emilgardis
Copy link
Member

can you show me docker version

is bin/docker actually docker or nerdctl? There might be an issue here were your docker binary is incorrectly actually docker and not containerd via nerdctl

@Emilgardis
Copy link
Member

if nerdctl is what is needed, you could use the env-var CROSS_CONTAINER_ENGINE=nerdctl

@secana
Copy link
Author

secana commented Aug 6, 2024

I think we are getting closer to the issue.

> docker version
Client:
 Version:           26.1.0-rd
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        cca8e72
 Built:             Tue Apr 23 16:25:03 2024
 OS/Arch:           darwin/arm64
 Context:           default
error during connect: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/version": EOF

> which docker
/Users/X/.rd/bin/docker

> which nerdctl
/Users/X/.rd/bin/nerdctl

It seems that Rancher installs nerdctl and containerd as the default but additionally docker + moby as it is possible to switch to those, too. In the nerdctl default mode, docker cli is available but docker is not running, so there is no sock to connect to. If cross checks if docker exits and tries to use it, it fails.

Telling cross to use nerdctl explicitly seems the right solution, if that's the issue. Unfortunately setting the env. var. leads to another error:

> export CROSS_CONTAINER_ENGINE=nerdctl
> cross build --target aarch64-unknown-linux-gnu
FATA[0000] unknown flag: --userns
Error: exit status 1

Btw, thanks for being super responsive @Emilgardis!

@Emilgardis
Copy link
Member

Emilgardis commented Aug 6, 2024 •
edited
Loading

ugh, --userns is not documented, thus nerdctl doesn't implement it :D (docker/docs#7071)

we should remove userns when engine is nerdctl, however for now you can do CROSS_CONTAINER_USER_NAMESPACE=none

@secana
Copy link
Author

secana commented Aug 6, 2024

Yeah! That starts to pull the image!

But I'm really sorry to bother you with the next issue :D

> export CROSS_CONTAINER_ENGINE=nerdctl
> export CROSS_CONTAINER_USER_NAMESPACE=none
> cross build --target aarch64-unknown-linux-gnu
ghcr.io/cross-rs/aarch64-unknown-linux-gnu:main:                                  resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:9808d7b7563a6334042c00555e473d0e0be46a171e3426301030118807280c86:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:168fa652629cedcd9549e85258be8f83fa008625b187c004d6ea439cf16f6a41: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:09d507265d4d68195761f92e4386ff2c59996d7b25bc8bab4a433f52d684f527:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:ec1ec5f628bd5c711a177303506e91f3cabeea17a9a99a4560427e0c6d3d36ac:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:2437fb9edcccd88fae58518f0a173701c3b76a003748306ff2f14ca587373bd4:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:9ea8908f47652b59b8055316d9c0e16b365e2b5cee15d3efcb79e2957e3e7cad:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:372e12503c26c04efde3b5ac5c8b48169444c25c4a29a51d71fe20f001bfb8b3:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:56bc9cd530688a4a1dd2205e9f3b92df9518c68abb9449e8c1bbe532d3436a46:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:f26e5a1a56aab880851c8acd969160322adc38286581b5b41eaf6e1763f57512:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:9f301d56d553b39fbc538e99b834b0d34dd6c718bb9d8875f343b7ba3328e548:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:4983db9bd2821e903a8521249618fcc7564fad14ecb1296f19c579fc852a1f6f:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:d2daf07ccb455dbc9baf0b58b4a9afba02f297dcfa4fcc88430de62b40ae6f48:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:20ccf0dd482e3e2110f45d1249699211cb58a318e30916aec3ceca967d075ffb:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:3822ee7993bdb2659e69053703b2e39146854328b049d4c0ee3d7e903b019479:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:47c0e240713a26fdde7313857795b4fe3233b8dbdde1d06ca324fdd84f5eb58a:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:9e5154ac9f3d3847a95c498c53c815fe2f43b04c49328589c068bfeff4791d89:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:c730e5e04ef635503a334bc044f2e35eeb046092fbf1f82888dece5028755b46:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:ed0a30f1e47185d95b906b75da08bba15ae2251cd4a490c52f11e22bd7ebd58a:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:c14f9f387e8ec83528cdfa29aec28da41abb5086ba244fcacf0fe0771b535474:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:f95d806fbd629f8a113d0132987e2ac9b0734aecbbbfaa4f7f683c630b8193ba:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:75e06a0297aa12adad112e95cfbb1bd7cefae376d927c37b901797614dd3e8fa:    done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:33779e87ffe4697ca66d852bbe0b5bebd68eff670aacb654554beb7df3a07768:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 52.7s                                                                    total:  551.3  (10.5 MiB/s)
WARN[0052] unsupported volume option "z"
WARN[0052] unsupported volume option "z"
WARN[0052] unsupported volume option "z"
WARN[0052] unsupported volume option "z"
WARN[0052] unsupported volume option "z"
FATA[0052] invalid name "cross-stable-x86_64-unknown-linux-gnu-96bfb-9b00956e5-aarch64-unknown-linux-gnu-00a65-1722935715050": identifier "cross-stable-x86_64-unknown-linux-gnu-96bfb-9b00956e5-aarch64-unknown-linux-gnu-00a65-1722935715050" greater than maximum length (76 characters): invalid argument
Error: exit status 1

@Emilgardis
Copy link
Member

Emilgardis commented Aug 6, 2024 •
edited
Loading

meh, that seems arbitrary.

you can rename the image pre-build uses by manually doing what it does...

#Cross.toml
[build]
image = "my-new-name"
dockerfile = "./Dockerfile"
#Dockerfile
ARG CROSS_BASE_IMAGE
FROM $CROSS_BASE_IMAGE
RUN <do stuff here>

kindof a hack but oh well

@Emilgardis
Copy link
Member

containerd/nerdctl#2942

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature-request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Emilgardis @secana