Portmapper improvements, and options to disable NAT #47871

robmry · 2024-05-28T19:55:48Z

- What I did

Allocate the same port for IPv4 and IPv6.
- For example:
  - use the same ephemeral port for 0.0.0.0 and :: in -p 80
  - use the same port for all addresses with matching port-mappings (same ranges and protocols):
    - -p 0.0.0.0:8080-8083:80 -p '[::]:8080-8083:80'
    - -p 0.0.0.0::80 -p '[::]::80'
    - -p 233.252.0.2::80 -p '[2001:DB8::2]::80
- If the same port can't be allocated for all addresses, fail with an error.
- If a host port is unexpectedly in-use, continue to look for an available port in the rest of the range.
- fixes Different port number are getting mapped on ipv4 and ipv6 for same expose port #42442
- fixes port ranges: docker picks a host port used by process other than itself #44136
Add bridge driver option com.docker.network.bridge.gateway_mode_ipv6=<nat|routed> (and com.docker.network.bridge.gateway_mode_ipv4=<nat|routed>).
- When set to routed, no NAT or masquerade iptables rules are set up.
- The container is still firewalled, with an open port for each -p.
- In inspect output, a non-NAT'd port mapping is shown with "HostPort": "".
- So, if the host's network has routing for a docker network, it's possible to communicate directly with the container's address.
- As with existing -p (HostConfig.PortBindings) options, a --expose (Config.ExposedPorts) for the container's port is still required to make the -p do anything. On its own, --expose doesn't result in the port being opened.
- Host address/ports in -p for a disabled-NAT network are ignored, the container's port is opened.

- How I did it

Replaced the v4/v6 portmappers with (simpler) bridge code.
Also moved the bridge-specific per-port iptables code into the bridge driver.

- How to verify it

New/updated tests.

# docker network create --ipv6 --subnet fdee::/64 -o com.docker.network.bridge.gateway_mode_ipv6=routed nnn
# netcat -l -p 8080   # make the first port in the range below unusable
# docker run --rm -ti --network nnn --name c1 -p 8080-8083:80 nginx
# docker inspect c1
[
    {
        ...
        "HostConfig": {
            ...
            "PortBindings": {
                "80/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "8080-8083"
                    }
                ]
            },
        "Config": {
            ...
            "ExposedPorts": {
                "80/tcp": {}
            },
        },
        "NetworkSettings": {
            ...
            "Ports": {
                "80/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "8081"
                    },
                    {
                        "HostIp": "::",
                        "HostPort": ""
                    }
                ]
            }
}}]

# IPv4 has NAT rules ...
*nat
-A POSTROUTING -s 172.19.0.2/32 -d 172.19.0.2/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -s 172.19.0.0/16 ! -o br-c68a9a829188 -j MASQUERADE
-A DOCKER ! -i br-c68a9a829188 -p tcp -m tcp --dport 8081 -j DNAT --to-destination 172.19.0.2:80
*filter
-A DOCKER -d 172.19.0.2/32 ! -i br-c68a9a829188 -o br-c68a9a829188 -p tcp -m tcp --dport 80 -j ACCEPT

# IPv6 has no NAT rules ...
*filter
-A DOCKER -d fdee::2/128 ! -i br-c68a9a829188 -o br-c68a9a829188 -p tcp -m tcp --dport 80 -j ACCEPT

# docker-proxy for IPv4 only ...
# ps -ef | grep docker-proxy
root     11894 11420  0 15:08 pts/0    00:00:00 /usr/local/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 8081 -container-ip 172.19.0.2 -container-port 80

On a "CentOS 9 (stream)" VM, running firewalld, with addresses 192.168.175.43 and fd48:70ba:de62:0:a00:27ff:fece:c637 ...

created a network with no-nat for IPv6 as above (with IPv6 subnet fd12:3456::/64).
Ran firewall-cmd --reload, checked that the rules were re-created.
From another host, added a static route to the VM and checked that 'http://[fd12:3456::2]' and curl 'http://192.168.175.43:8080' worked as expected.

- Description for the changelog

- In port mappings with no host port specified, and with host port ranges, allocate the same port for IPv4 and IPv6.
- Add bridge driver option `com.docker.network.bridge.gateway_mode_ipv6=<nat|routed>`, when
  set to `routed` no NAT or masquerading rules are configured for port mappings.
  - This enables direct IPv6 access to the container, if the host's network can route packets for the
     container's address to the host. Mapped ports will be opened in the containers firewall.
  - For example, `docker network create --ipv6 -o com.docker.network.bridge.gateway_mode_ipv6=routed mynet`.
  - Option `com.docker.network.bridge.gateway_mode_ipv4=<nat|routed>` is also available.

libnetwork/drivers/bridge/port_mapping_linux.go

akerouanton · 2024-05-30T14:52:16Z

libnetwork/portallocator/portallocator.go

 	p.mutex.Lock()
 	defer p.mutex.Unlock()

-	if ip == nil {


Why remove this?

It's never called with ip == nil, and removing the check makes the code simpler. But, I didn't do the same for RequestPortInRange, so I can put it back.

It's never called with ip == nil

Oh, right! Then, maybe remove that in a follow-up commit / PR.

libnetwork/portallocator/portallocator.go

libnetwork/drivers/bridge/bridge_linux_test.go

akerouanton · 2024-06-03T14:44:37Z

libnetwork/drivers/bridge/bridge_store.go

+	// on live-restore we don't want to allocate different ports - so, remove the range
+	// from the operational data.
+	for _, pm := range ep.portMapping {
+		pm.HostPortEnd = pm.HostPort


I'm wondering if this is really the right place to do that. Based on your comment, it seems we mix state and config in a struct meant to store only the end state. Maybe this should be done when ep.portMapping entries are created instead.

This is dealing with old saved-state - it represents running config, the ports can't be a range, but that's how the old code stored it.

The new code will treat it as a range on live-restore - it won't know it's live-restore, it's just been given the struct as config as if it had come from the API.

So the old saved-state has to be fixed up ... where else could it be done?

This is dealing with old saved-state

Oh, right 🙈 Maybe we should consider dropping this compatibility code after a few releases.

I've added a comment to say it can be removed when upgrade from 26.x is no longer supported.

akerouanton · 2024-06-03T18:49:20Z

libnetwork/drivers/bridge/port_mapping_linux.go

+	} else if b.HostIP.To4() != nil {
+		v = iptables.IPv4
+	} else {
+		// The binding is between containerV4 and hostV6, handled by docker-proxy


Since IPv4-mapped IPv6 addrs aren't properly unmapped, I think this condition doesn't hold true.

Overall I think this condition is a bit cursed since each branch doesn't test the same field. I would rewrite it to explicitly early return whenever b.IP and b.HostIp aren't of the same family -- but then net's lack of expressiveness kicks in (ie. you'd have to write a helper function).

As above, I'll take a look at the handling of IPv4-mapped IPv6.

akerouanton · 2024-06-03T18:55:20Z

libnetwork/drivers/bridge/port_mapping_linux.go

-	if bnd.HostIP.To4() == nil {
-		portmapper = n.portMapperV6
+func (n *bridgeNetwork) reapplyPerPortIptables4() {
+	n.reapplyPerPortIptables(func(b *portBinding) bool { return b.IP.To4() != nil })


Oh, that's contrived! Better use an arg that clearly states what address family should be restored, and then let it filter port bindings based on that.

We should in a follow-up if we can just restore everything in a single pass and call iptables.OnReloaded only once.

Passing in a filter function is contrived? Better than a bool ipv4 or similar I think. But, can do that.

libnetwork/portmapper/mapper.go

libnetwork/drivers/bridge/port_mapping_linux.go

libnetwork/iptables/iptables_test.go

corhere

All this talk about IPv6-mapped-IPv4 addresses is making me nervous. There is no semantic difference between listen()ing or bind()ing on [::ffff:10.0.0.1]:8080 vs. 10.0.0.1:8080. Only IPv4 packets will be accepted/transmitted over the wire for such a socket. So why does the portmapper need to fuss so much about net.IP slice lengths?

libnetwork/portallocator/portallocator.go

corhere · 2024-06-04T19:41:25Z

libnetwork/portallocator/portallocator.go

+		ipstr := ip.String()
+		if _, ok := p.ipMap[ipstr]; !ok {


Have you considered converting the implementation to use net/netip types internally, or would it be a bit much to tack onto this PR?

Yes, it'd be better - but also yes, a bit too much for this PR.

libnetwork/portallocator/portallocator.go

libnetwork/portmapper/proxy.go

corhere · 2024-06-04T20:10:16Z

libnetwork/drivers/bridge/bridge_linux.go

+	n.Lock()
+	defer n.Unlock()
+	if n.driver == nil {
+		return ""
+	}


What's with the locking and defensive coding? It looks a lot like n.driver is guaranteed to be non-nil and immutable.

I think you're right - but every other 'bridgeNetwork method makes the same checks. A separate review/rework of all that could be good. But, for this change, it seems best to stick with tradition.

libnetwork/drivers/bridge/port_mapping_linux.go

libnetwork/iptables/iptables_test.go

integration/internal/container/ops.go

libnetwork/drivers/bridge/port_mapping_linux.go

robmry · 2024-06-05T14:50:45Z

All this talk about IPv6-mapped-IPv4 addresses is making me nervous. There is no semantic difference between listen()ing or bind()ing on [::ffff:10.0.0.1]:8080 vs. 10.0.0.1:8080. Only IPv4 packets will be accepted/transmitted over the wire for such a socket. So why does the portmapper need to fuss so much about net.IP slice lengths?

It needs to put rules into iptables or ip6tables, so it has to know what sort of address it's really dealing with.

libnetwork/portmapper/proxy_linux.go

libnetwork/drivers/bridge/port_mapping_linux_test.go

libnetwork/drivers/bridge/port_mapping_linux.go

Add bridge driver options... com.docker.network.bridge.gateway_mode_ipv4=<nat|routed> com.docker.network.bridge.gateway_mode_ipv6=<nat|routed> If set to "routed", no NAT or masquerade rules are set up for port mappings. When NAT is disabled, the mapping is shown in 'inspect' output with no host port number. For example, for "-p 80" with NAT disabled for IPv6 but not IPv4: "80/tcp": [ { "HostIp": "0.0.0.0", "HostPort": "32768" }, { "HostIp": "::", "HostPort": "" } Signed-off-by: Rob Murray <[email protected]>

Without this change, if a port mapping did not specify a host address and the network was IPv6-enabled, the same port would be allocated for mappings from '0.0.0.0' and '::'. But, if the port mapping was specified with explicit addresses even, for example: -p 0.0.0.0:8080-8083:80 -p '[::]:8083-8080:80' This change looks for port mappings that only differ in the host IP address, and makes sure it allocates the same port for all of them. If it can't, it fails with an error. Signed-off-by: Rob Murray <[email protected]>

It was added so that tests could replace it before it was picked up and used by a new network's PortMapper, so that tests were isolated from each other. Now the PortMapper is not used by the bridge driver, neither is driver's portAllocator. Instead of replacing the driver.portAllocator in tests, reset the singleton instance using its ReleaseAll(). Un-export portallocator.NewInstance, now the tests aren't using it. Signed-off-by: Rob Murray <[email protected]>

Display a PortBinding in a format that's more like the one used in the CLI, but includes the container IP if known. Signed-off-by: Rob Murray <[email protected]>

When bridge driver opt com.docker.network.bridge.gatway_mode_ipv[46] is set to "routed", there is no NAT. When there's no NAT, there's no meaning to the HostPort field in a port mapping (all the port mapping does is open the container's port), and the HostIP field is only used to determine the address family. So, check port bindings, and raise errors if fields are unexpectedly set when the mapping only applies to a gateway_mode=routed network. Zero-addresses are allowed, to say the mapping/open-port should be IPv4-only or IPv6-only, and host ports are not allowed. A mapping with no host address, so it applies to IPv4 and IPv6 when the default binding is 0.0.0.0, may include a host port if either uses NAT. The port number is ignored for the directly-routed family. Signed-off-by: Rob Murray <[email protected]>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [docker/docker](https://togithub.com/docker/docker) | major | `26.1.4` -> `27.0.3` | --- ### Release Notes <details> <summary>docker/docker (docker/docker)</summary> ### [`v27.0.3`](https://togithub.com/moby/moby/releases/tag/v27.0.3) [Compare Source](https://togithub.com/docker/docker/compare/v27.0.2...v27.0.3) #### 27.0.3 For a full list of pull requests and changes in this release, refer to the relevant GitHub milestones: - [docker/cli, 27.0.3 milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A27.0.3) - [moby/moby, 27.0.3 milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A27.0.3) - Deprecated and removed features, see [Deprecated Features](https://togithub.com/docker/cli/blob/v27.0.3/docs/deprecated.md). - Changes to the Engine API, see [API version history](https://togithub.com/moby/moby/blob/v27.0.3/docs/api/version-history.md). ##### Bug fixes and enhancements - Fix a regression that incorrectly reported a port mapping from a host IPv6 address to an IPv4-only container as an error. [moby/moby#48090](https://togithub.com/moby/moby/pull/48090) - Fix a regression that caused duplicate subnet allocations when creating networks. [moby/moby#48089](https://togithub.com/moby/moby/pull/48089) - Fix a regression resulting in "fail to register layer: failed to Lchown" errors when trying to pull an image with rootless enabled on a system that supports native overlay with user-namespaces. [moby/moby#48086](https://togithub.com/moby/moby/pull/48086) ### [`v27.0.2`](https://togithub.com/moby/moby/releases/tag/v27.0.2) [Compare Source](https://togithub.com/docker/docker/compare/v27.0.1-rc.1...v27.0.2) #### 27.0.2 For a full list of pull requests and changes in this release, refer to the relevant GitHub milestones: - [docker/cli, 27.0.2 milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A27.0.2) - [moby/moby, 27.0.2 milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A27.0.2) - Deprecated and removed features, see [Deprecated Features](https://togithub.com/docker/cli/blob/v27.0.2/docs/deprecated.md). - Changes to the Engine API, see [API version history](https://togithub.com/moby/moby/blob/v27.0.2/docs/api/version-history.md). ##### Bug fixes and enhancements - Fix a regression that caused port numbers to be ignored when parsing a Docker registry URL. [docker/cli#5197](https://togithub.com/docker/cli/pull/5197), [docker/cli#5198](https://togithub.com/docker/cli/pull/5198) ##### Removed - api/types: deprecate `ContainerJSONBase.Node` field and `ContainerNode` type. These definitions were used by the standalone ("classic") Swarm API, but never implemented in the Docker Engine itself. [moby/moby#48055](https://togithub.com/moby/moby/pull/48055) ### [`v27.0.1`](https://togithub.com/moby/moby/releases/tag/v27.0.1) [Compare Source](https://togithub.com/docker/docker/compare/v26.1.4...v27.0.1-rc.1) #### 27.0.1 For a full list of pull requests and changes in this release, refer to the relevant GitHub milestones: - [docker/cli, 27.0.0 milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A27.0.0) - [moby/moby, 27.0.0 milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A27.0.0) - Deprecated and removed features, see [Deprecated Features](https://togithub.com/docker/cli/blob/v27.0.1/docs/deprecated.md). - Changes to the Engine API, see [API version history](https://togithub.com/moby/moby/blob/v27.0.1/docs/api/version-history.md). ##### New - containerd image store: Add `--platform` flag to `docker image push` and improve the default behavior when not all platforms of the multi-platform image are available locally. [docker/cli#4984](https://togithub.com/docker/cli/pull/4984), [moby/moby#47679](https://togithub.com/moby/moby/pull/47679) - Add support to `docker stack deploy` for `driver_opts` in a service's networks. [docker/cli#5125](https://togithub.com/docker/cli/pull/5125) - Consider additional `/usr/local/libexec` and `/usr/libexec` paths when looking up the userland proxy binaries by a name with a `docker-` prefix. [moby/moby#47804](https://togithub.com/moby/moby/pull/47804) ##### Bug fixes and enhancements - `*client.Client` instances are now always safe for concurrent use by multiple goroutines. Previously, this could lead to data races when the `WithAPIVersionNegotiation()` option is used. [moby/moby#47961](https://togithub.com/moby/moby/pull/47961) - Fix a bug causing the Docker CLI to leak Unix sockets in `$TMPDIR` in some cases. [docker/cli#5146](https://togithub.com/docker/cli/pull/5146) - Don't ignore a custom seccomp profile when used in conjunction with `--privileged`. [moby/moby#47500](https://togithub.com/moby/moby/pull/47500) - rootless: overlay2: support native overlay diff when using rootless-mode with Linux kernel version 5.11 and later. [moby/moby#47605](https://togithub.com/moby/moby/pull/47605) - Fix the `StartInterval` default value of healthcheck to reflect the documented value of 5s. [moby/moby#47799](https://togithub.com/moby/moby/pull/47799) - Fix `docker save` and `docker load` not ending on the daemon side when the operation was cancelled by the user, for example with <kbd>Ctrl+C</kbd>. [moby/moby#47629](https://togithub.com/moby/moby/pull/47629) - The `StartedAt` property of containers is now recorded before container startup, guaranteeing that the `StartedAt` is always before `FinishedAt`. [moby/moby#47003](https://togithub.com/moby/moby/pull/47003) - The internal DNS resolver used by Windows containers on Windows now forwards requests to external DNS servers by default. This enables `nslookup` to resolve external hostnames. This behaviour can be disabled via `daemon.json`, using `"features": { "windows-dns-proxy": false }`. The configuration option will be removed in a future release. [moby/moby#47826](https://togithub.com/moby/moby/pull/47826) - Print a warning when the CLI does not have permissions to read the configuration file. [docker/cli#5077](https://togithub.com/docker/cli/pull/5077) - Fix a goroutine and file-descriptor leak on container attach. [moby/moby#45052](https://togithub.com/moby/moby/pull/45052) - Clear the networking state of all stopped or dead containers during daemon start-up. [moby/moby#47984](https://togithub.com/moby/moby/pull/47984) - Write volume options JSON atomically to avoid "invalid JSON" errors after system crash. [moby/moby#48034](https://togithub.com/moby/moby/pull/48034) - Allow multiple macvlan networks with the same parent. [moby/moby#47318](https://togithub.com/moby/moby/pull/47318) - Allow BuildKit to be used on Windows daemons that advertise it. [docker/cli#5178](https://togithub.com/docker/cli/pull/5178) ##### Networking - Allow sysctls to be set per-interface during container creation and network connection. [moby/moby#47686](https://togithub.com/moby/moby/pull/47686) - In a future release, this will be the only way to set per-interface sysctl options. For example, on the command line in a `docker run` command,`--network mynet --sysctl net.ipv4.conf.eth0.log_martians=1` will be rejected. Instead, you must use `--network name=mynet,driver-opt=com.docker.network.endpoint.sysctls=net.ipv4.conf.IFNAME.log_martians=1`. ##### IPv6 - `ip6tables` is no longer experimental. You may remove the `experimental` configuration option and continue to use IPv6, if it is not required by any other features. - `ip6tables` is now enabled for Linux bridge networks by default. [moby/moby#47747](https://togithub.com/moby/moby/pull/47747) - This makes IPv4 and IPv6 behaviors consistent with each other, and reduces the risk that IPv6-enabled containers are inadvertently exposed to the network. - There is no impact if you are running Docker Engine with `ip6tables` enabled (new default). - If you are using an IPv6-enabled bridge network without `ip6tables`, this is likely a breaking change. Only published container ports (`-p` or `--publish`) are accessible from outside the Docker bridge network, and outgoing connections masquerade as the host. - To restore the behavior of earlier releases, no `ip6tables` at all, set `"ip6tables": false` in `daemon.json`, or use the CLI option `--ip6tables=false`. Alternatively, leave `ip6tables` enabled, publish ports, and enable direct routing. - With `ip6tables` enabled, if `ip6tables` is not functional on your host, Docker Engine will start but it will not be possible to create an IPv6-enabled network. ##### IPv6 network configuration improvements - A Unique Local Address (ULA) base prefix is automatically added to `default-address-pools` if this parameter wasn't manually configured, or if it contains no IPv6 prefixes. [moby/moby#47853](https://togithub.com/moby/moby/pull/47853) - Prior to this release, to create an IPv6-enabled network it was necessary to use the `--subnet` option to specify an IPv6 subnet, or add IPv6 ranges to `default-address-pools` in `daemon.json`. - Starting in this release, when a bridge network is created with `--ipv6` and no IPv6 subnet is defined by those options, an IPv6 Unique Local Address (ULA) base prefix is used. - The ULA prefix is derived from the Engine host ID such that it's unique across hosts and over time. - IPv6 address pools of any size can now be added to `default-address-pools`. [moby/moby#47768](https://togithub.com/moby/moby/pull/47768) - IPv6 can now be enabled by default on all custom bridge networks using `"default-network-opts": { "bridge": {"com.docker.network.enable_ipv6": "true"}}` in `daemon.json`, or `dockerd --default-network-opt=bridge=com.docker.network.enable_ipv6=true`on the comand line. [moby/moby#47867](https://togithub.com/moby/moby/pull/47867) - Direct routing for IPv6 networks, with `ip6tables` enabled. [moby/moby#47871](https://togithub.com/moby/moby/pull/47871) - Added bridge driver option `com.docker.network.bridge.gateway_mode_ipv6=<nat|routed>`. - The default behavior, `nat`, is unchanged from previous releases running with `ip6tables` enabled. NAT and masquerading rules are set up for each published container port. - When set to `routed`, no NAT or masquerading rules are configured for published ports. This enables direct IPv6 access to the container, if the host's network can route packets for the container's address to the host. Published ports will be opened in the container's firewall. - When a port mapping only applies to `routed` mode, only addresses `0.0.0.0` or `::` are allowed and a host port must not be given. - Note that published container ports, in `nat` or `routed` mode, are accessible from any remote address if routing is set up in the network, unless the Docker host's firewall has additional restrictions. For example: `docker network create --ipv6 -o com.docker.network.bridge.gateway_mode_ipv6=routed mynet`. - The option `com.docker.network.bridge.gateway_mode_ipv4=<nat|routed>` is also available, with the same behavior but for IPv4. - If firewalld is running on the host, Docker creates policy `docker-forwarding` to allow forwarding from any zone to the `docker` zone. This makes it possible to configure a bridge network with a routable IPv6 address, and no NAT or masquerading. [moby/moby#47745](https://togithub.com/moby/moby/pull/47745) - When a port is published with no host port specified, or a host port range is given, the same port will be allocated for IPv4 and IPv6. [moby/moby#47871](https://togithub.com/moby/moby/pull/47871) - For example `-p 80` will result in the same ephemeral port being allocated for `0.0.0.0` and `::`, and `-p 8080-8083:80` will pick the same port from the range for both address families. - Similarly, ports published to specific addresses will be allocated the same port. For example, `-p 127.0.0.1::80 -p '[::1]::80'`. - If no port is available on all required addresses, container creation will fail. - Environment variable `DOCKER_ALLOW_IPV6_ON_IPV4_INTERFACE`, introduced in release 26.1.1, no longer has any effect. [moby/moby#47963](https://togithub.com/moby/moby/pull/47963) - If IPv6 could not be disabled on an interface because of a read-only `/proc/sys/net`, the environment variable allowed the container to start anyway. - In this release, if IPv4 cannot be disabled for an interface, IPv6 can be explicitly enabled for the network simply by using `--ipv6` when creating it. Other workarounds are to configure the OS to disable IPv6 by default on new interfaces, mount `/proc/sys/net` read-write, or use a kernel with no IPv6 support. - For IPv6-enabled bridge networks, do not attempt to replace the bridge's kernel-assigned link local address with `fe80::1`. [moby/moby#47787](https://togithub.com/moby/moby/pull/47787) ##### Removed - Deprecate experimental GraphDriver plugins. [moby/moby#48050](https://togithub.com/moby/moby/pull/48050), [docker/cli#5172](https://togithub.com/docker/cli/pull/5172) - pkg/archive: deprecate `NewTempArchive` and `TempArchive`. These types were only used in tests and will be removed in the next release. [moby/moby#48002](https://togithub.com/moby/moby/pull/48002) - pkg/archive: deprecate `CanonicalTarNameForPath` [moby/moby#48001](https://togithub.com/moby/moby/pull/48001) - Deprecate pkg/dmesg. This package was no longer used, and will be removed in the next release. [moby/moby#47999](https://togithub.com/moby/moby/pull/47999) - Deprecate `pkg/stringid.ValidateID` and `pkg/stringid.IsShortID` [moby/moby#47995](https://togithub.com/moby/moby/pull/47995) - runconfig: deprecate `SetDefaultNetModeIfBlank` and move `ContainerConfigWrapper` to `api/types/container` [moby/moby#48007](https://togithub.com/moby/moby/pull/48007) - runconfig: deprecate `DefaultDaemonNetworkMode` and move to `daemon/network` [moby/moby#48008](https://togithub.com/moby/moby/pull/48008) - runconfig: deprecate `opts.ConvertKVStringsToMap`. This utility is no longer used, and will be removed in the next release. [moby/moby#48016](https://togithub.com/moby/moby/pull/48016) - runconfig: deprecate `IsPreDefinedNetwork`. [moby/moby#48011](https://togithub.com/moby/moby/pull/48011) ##### API - containerd image store: `POST /images/{name}/push` now supports a `platform` parameter (JSON encoded OCI Platform type) that allows selecting a specific platform-manifest from the multi-platform image. This is experimental and may change in future API versions. [moby/moby#47679](https://togithub.com/moby/moby/pull/47679) - `POST /services/create` and `POST /services/{id}/update` now support `OomScoreAdj`. [moby/moby#47950](https://togithub.com/moby/moby/pull/47950) - `ContainerList` api returns container annotations. [moby/moby#47866](https://togithub.com/moby/moby/pull/47866) - `POST /containers/create` and `POST /services/create` now take `Options` as part of `HostConfig.Mounts.TmpfsOptions` allowing to set options for tmpfs mounts. [moby/moby#46809](https://togithub.com/moby/moby/pull/46809) - The `Healthcheck.StartInterval` property is now correctly ignored when updating a Swarm service using API versions less than v1.44. [moby/moby#47991](https://togithub.com/moby/moby/pull/47991) - `GET /events` now supports image `create` event that is emitted when a new image is built regardless if it was tagged or not. [moby/moby#47929](https://togithub.com/moby/moby/pull/47929) - `GET /info` now includes a `Containerd` field containing information about the location of the containerd API socket and containerd namespaces used by the daemon to run containers and plugins. [moby/moby#47239](https://togithub.com/moby/moby/pull/47239) - Deprecate non-standard (config) fields in image inspect output. The `Config` field returned by this endpoint (used for `docker image inspect`) returned additional fields that are not part of the image's configuration and not part of the [Docker Image Spec] and the [OCI Image Spec]. These fields are never set (and always return the default value for the type), but are not omitted in the response when left empty. As these fields were not intended to be part of the image configuration response, they are deprecated, and will be removed in the future API versions. - Deprecate the daemon flag `--api-cors-header` and the corresponding `daemon.json` configuration option. These will be removed in the next major release. [moby/moby#45313](https://togithub.com/moby/moby/pull/45313) The following deprecated fields are currently included in the API response, but are not part of the underlying image's `Config`: [moby/moby#47941](https://togithub.com/moby/moby/pull/47941) - `Hostname` - `Domainname` - `AttachStdin` - `AttachStdout` - `AttachStderr` - `Tty` - `OpenStdin` - `StdinOnce` - `Image` - `NetworkDisabled` (already omitted unless set) - `MacAddress` (already omitted unless set) - `StopTimeout` (already omitted unless set) ##### Go SDK changes - Client API callback for the following functions now require a context parameter. [moby/moby#47536](https://togithub.com/moby/moby/pull/47536) - `client.RequestPrivilegeFunc` - `client.ImageSearchOptions.AcceptPermissionsFunc` - `image.ImportOptions.PrivilegeFunc` - Remove deprecated aliases for Image types. [moby/moby#47900](https://togithub.com/moby/moby/pull/47900) - `ImageImportOptions` - `ImageCreateOptions` - `ImagePullOptions` - `ImagePushOptions` - `ImageListOptions` - `ImageRemoveOptions` - Introduce `Ulimit` type alias for `github.com/docker/go-units.Ulimit`. The `Ulimit` type as used in the API is defined in a Go module that will transition to a new location in future. A type alias is added to reduce the friction that comes with moving the type to a new location. The alias makes sure that existing code continues to work, but its definition may change in future. Users are recommended to use this alias instead of the `units.Ulimit` directly. [moby/moby#48023](https://togithub.com/moby/moby/pull/48023) - Move and rename types, changing their import paths and exported names. [moby/moby#47936](https://togithub.com/moby/moby/pull/47936), [moby/moby#47873](https://togithub.com/moby/moby/pull/47873), [moby/moby#47887](https://togithub.com/moby/moby/pull/47887), [moby/moby#47882](https://togithub.com/moby/moby/pull/47882), [moby/moby#47921](https://togithub.com/moby/moby/pull/47921), [moby/moby#48040](https://togithub.com/moby/moby/pull/48040): - Move the following types to `api/types/container`: - `BlkioStatEntry` - `BlkioStats` - `CPUStats` - `CPUUsage` - `ContainerExecInspect` - `ContainerPathStat` - `ContainerStats` - `ContainersPruneReport` - `CopyToContainerOptions` - `ExecConfig` - `ExecStartCheck` - `MemoryStats` - `NetworkStats` - `PidsStats` - `StatsJSON` - `Stats` - `StorageStats` - `ThrottlingData` - Move the following types to `api/types/image`: - `ImagesPruneReport` - `ImageImportSource` - `ImageLoadResponse` - Move the `ExecStartOptions` type to `api/types/backend`. - Move the `VolumesPruneReport` type to `api/types/volume`. - Move the `EventsOptions` type to `api/types/events`. - Move the `ImageSearchOptions` type to `api/types/registry`. - Drop `Network` prefix and move the following types to `api/types/network`: - `NetworkCreateResponse` - `NetworkConnect` - `NetworkDisconnect` - `NetworkInspectOptions` - `EndpointResource` - `NetworkListOptions` - `NetworkCreateOptions` - `NetworkCreateRequest` - `NetworksPruneReport` - Move `NetworkResource` to `api/types/network`. ##### Packaging updates - Update Buildx to [v0.15.1](https://togithub.com/docker/buildx/releases/tag/v0.15.1). [docker/docker-ce-packaging#1029](https://togithub.com/docker/docker-ce-packaging/pull/1029) - Update BuildKit to [v0.14.1](https://togithub.com/moby/buildkit/releases/tag/v0.14.1). [moby/moby#48028](https://togithub.com/moby/moby/pull/48028) - Update runc to [v1.1.13](https://togithub.com/opencontainers/runc/releases/tag/v1.1.13) [moby/moby#47976](https://togithub.com/moby/moby/pull/47976) - Update Compose to [v2.28.1](https://togithub.com/docker/compose/releases/tag/v2.28.1). [moby/docker-ce-packaging#1032](https://togithub.com/docker/docker-ce-packaging/pull/1032) [Docker image spec]: https://togithub.com/moby/docker-image-spec/blob/v1.3.1/specs-go/v1/image.go#L19-L32 [OCI Image Spec]: https://togithub.com/opencontainers/image-spec/blob/v1.1.0/specs-go/v1/config.go#L24-L62 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6am on monday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/earthly/dind).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

robmry force-pushed the portmapper_fixes_and_nonat branch from 0770137 to 68c6534 Compare May 29, 2024 14:22

robmry changed the title ~~Portmapper fixes and nonat~~ Portmapper improvements, and options to disable NAT May 29, 2024

robmry force-pushed the portmapper_fixes_and_nonat branch from 68c6534 to d594859 Compare May 29, 2024 16:11

robmry self-assigned this May 29, 2024

robmry added this to the 27.0.0 milestone May 29, 2024

robmry requested review from akerouanton and corhere May 29, 2024 16:14

robmry marked this pull request as ready for review May 29, 2024 16:25

robmry mentioned this pull request May 29, 2024

Disable bridge n/w masquerading for IPv4, IPv6, or both. #47786

Closed

robmry commented May 30, 2024

View reviewed changes

libnetwork/drivers/bridge/port_mapping_linux.go Outdated Show resolved Hide resolved

robmry force-pushed the portmapper_fixes_and_nonat branch from d594859 to e066f01 Compare May 30, 2024 18:05

akerouanton reviewed Jun 3, 2024

View reviewed changes

robmry force-pushed the portmapper_fixes_and_nonat branch 2 times, most recently from e61b9db to 026c263 Compare June 4, 2024 15:55

corhere reviewed Jun 4, 2024

View reviewed changes

robmry force-pushed the portmapper_fixes_and_nonat branch from 026c263 to bc962e6 Compare June 5, 2024 14:45

robmry force-pushed the portmapper_fixes_and_nonat branch from bc962e6 to f6cbc15 Compare June 5, 2024 15:21

corhere reviewed Jun 5, 2024

View reviewed changes

robmry force-pushed the portmapper_fixes_and_nonat branch 2 times, most recently from 8ce32a2 to 4c0e873 Compare June 5, 2024 22:25

This was referenced Jun 11, 2024

docker-proxy accepts connections before NAT rules are set up #47951

Closed

Remove iptables rule for SCTP checksum fixup #47952

Closed

Use "hairpin mode" by-default / more often #47953

Open

robmry added 5 commits June 11, 2024 22:33

Improve PortBinding.String()

2a291c1

Display a PortBinding in a format that's more like the one used in the CLI, but includes the container IP if known. Signed-off-by: Rob Murray <[email protected]>

robmry force-pushed the portmapper_fixes_and_nonat branch from 8c2e4ca to 01eecb6 Compare June 11, 2024 21:34

robmry mentioned this pull request Jun 12, 2024

Updates for moby 27.0 docker/docs#20026

Merged

akerouanton approved these changes Jun 12, 2024

View reviewed changes

corhere approved these changes Jun 12, 2024

View reviewed changes

robmry merged commit 52333f3 into moby:master Jun 13, 2024

robmry deleted the portmapper_fixes_and_nonat branch June 13, 2024 08:14

robmry mentioned this pull request Jun 18, 2024

Fix hairpin between networks with mapped port #48025

Merged

felixfontein mentioned this pull request Jun 27, 2024

CI: Skip certain tests on Docker 27.0.0+ ansible-collections/community.docker#893

Merged

thaJeztah mentioned this pull request Jun 28, 2024

NAT is disabled error when setting IPV6 loopback address in port mapping #48067

Closed

robmry mentioned this pull request Jun 28, 2024

Fix incorrect validation of port mapping #48088

Merged

thaJeztah mentioned this pull request Jun 28, 2024

[27.0 backport] Fix incorrect validation of port mapping #48090

Merged

robmry mentioned this pull request Jul 4, 2024

Create docker-proxy TCP/UDP listener sockets in the daemon #48132

Merged

HofmeisterAn mentioned this pull request Aug 2, 2024

[Enhancement]: Resolve port bindings according to IPv4 and IPv6 testcontainers/testcontainers-dotnet#825

Closed

mdelapenya mentioned this pull request Aug 6, 2024

chore: resolve IPv4-IPv6 issues testcontainers/testcontainers-go#2403

Closed

kiview mentioned this pull request Aug 13, 2024

Error: No host port found for host IP testcontainers/testcontainers-node#818

Closed

thaJeztah mentioned this pull request Aug 15, 2024

[27.x backport] Create docker-proxy TCP/UDP listener sockets in the daemon #48338

Closed

kiview mentioned this pull request Oct 16, 2024

[Enhancement]: Improve port mapping consistency for IPv4 and IPv6 when using getMappedPort() method testcontainers/testcontainers-java#9411

Closed

robmry mentioned this pull request Nov 28, 2024

Add gateway mode "nat-unprotected" #48597

Merged

cristianrgreco mentioned this pull request Apr 14, 2025

Port mapping for ipv4 & ipv6 can cause different ports to be mapped and getMappedPort function will return wrong port testcontainers/testcontainers-node#750

Open

robmry mentioned this pull request Jun 5, 2025

Always allow access to published ports on addresses in gateway mode "routed" networks #50140

Merged

Portmapper improvements, and options to disable NAT #47871

Portmapper improvements, and options to disable NAT #47871

Uh oh!

Conversation

robmry commented May 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

corhere left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

robmry commented Jun 5, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

robmry commented May 28, 2024 •

edited

Loading