Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.51%. Comparing base (f67ab9e) to head (af0a190).
⚠️ Report is 6 commits behind head on master.

@@           Coverage Diff           @@
##           master   #16459   +/-   ##
=======================================
  Coverage   98.51%   98.51%           
=======================================
  Files         177      177           
  Lines       22992    22992           
=======================================
  Hits        22651    22651           
  Misses        341      341           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Code size report:

Reference:  docs/library/machine: Add port availability notes. [ff2ed49]
Comparison: extmod: Add USB Network (NCM) driver implementation. [merge of af0a190]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
      esp32:    +0 +0.000% ESP32_GENERIC
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

Interesting, is the assumption that the MCU is network capable, and will act as a bridge/router?

With this feature any MCU with TinyUSB based USB connection becomes network capable.

I hadn't thought of routing or bridging to other interfaces, though that's probably possible.

My initial use case at Planet Innovation is for larger systems where we have a Linux or windows host PC with an embedded module running micropython attached. We wanted an option to use standard networking protocols to communicate between the application on the host computer and the device, however not every system warrants an MCU with an Ethernet connection, or the host computer Ethernet is used for external connection and we want the internal connection on a separate interface etc.

With this feature the connection between host and device is still point to point like any USB connection, but you can easily run multiple services like a http server, mqtt client, webrepl etc.

Currently I don't think you can monitor or query for the IP address/s handed out by the micropython dhcp server to active clients, I might want to add that if I'm correct so that any client library could easily find / talk to services running on the host.

With this feature any MCU with TinyUSB based USB connection becomes network capable.

In that respect, who is server, who is client?

• Does a MP board without network interface get network capable as a client and uses a service provided by the entity it connects to? In that case, how is the service provided? or
• Is the MP board a server providing network services to client connected by USB? In that case the board must have already a network interface.

who is server, who is client?

There's no hard and fast rule here, both can run servers and be clients.

This is just a network connection.

Both the micropython device and the PC have their own network stacks.

The USB network connection kind of acts the same as if the micropython board has an Ethernet port that is connected to a PC Ethernet port via a crossover cable. So both ends have their own IP address that can be manually or automatically assigned but there's no router or hub in between.

Currently the micropython end is hardcoded to 192.168.7.1 though I'd plan to make that configurable eventually.

I have enabled the built-in dhcp server on micropython on the interface (so mpy end is a server from that perspective) and the PC end seems to get provisioned as 192.168.7.16 by default, though this isn't necessarily guaranteed.

If the PC happens to be running a web server then a micropython requests library will be able to access it on that 192.168.7.16 address.

Conversely if you run microdot or similar on the micropython board the PC will be able to access it on 192.168.7.1

I'm planning on often having a Linux host running mosquito mqtt broker which both the micropython board and the host PC can both publish/subscribe to. I've also looked briefly at a pure python mqtt broker that I might try running on the device instead though!

My point is server/client relationship is generally on a per-service basis and can work both ways.

This has been updated to make mdns work so the device can be pinged from host based on hostname, though it seems pretty slow to resolve initially on windows.

The ip addresses used by default are now in the link-local range which feels appropriate for this kind of isolated point-to-point link and should reduce the chance of ip collisions with other networks on the host.

Once the host is provided with an ip from the dhcp server in micropython, the host ip address is set as the gateway address on the micropython side of the interface which makes it easy for the micropython application to find/use the host ip address.

I've written up a quick demo of host to use mqtt between the device and host pc here, it works really well! https://github.com/andrewleech/mpy-usb-net-mqtt-demo/

Could you please introduce an example that forwards WLAN to RNDIS using this? It'd be a welcome thing in the usb examples folder.

EDIT: I was unable to build the branch (for rp2 ofc), I get this error:

ERROR: Cannot plug gap greater than alignment - gap 253892, alignment 32

EDIT: SOLVED: I forgot to properly specify my board variant and build variant.

EDIT: Nope, I don't get it, I've been trying for a while an am unable to build it for the Raspberry Pi Pico 2 W.

I tried with:

]~/Documents/ItAll/micropython/ports/rp2@ cmake -DMICROPY_BOARD=RPI_PICO2_W -DMICROPY_USB_NET=true
]~/Documents/ItAll/micropython/ports/rp2@ make -j4 -l4 BOARD=RPI_PICO2_W MICROPY_PY_USBNET=true
# Then I moved the firmware.uf2 file to my Pi
# Then I connected to it via serial after a reboot, and imported `network`, but wasn't able to find the USBNET interface.
# The network_usbd_ncm.c was built tho, as it appears in the `make` output

In case anyone feels like helping out a newbie. Thx.
I hope this gets merged soon :)

On most ports, features are generally enabled like:

make -j BOARD=RPI_PICO2_W CFLAGS_EXTRA="-DMICROPY_HW_NETWORK_USBNET=1"

That plug gap alignment error is a strange one, I actually got it yesterday on a RPI build on an unrelated branch. I think it's related to the compilation of picotool more than the firmware issue, don't know where it came.

I haven't used this change on a board with wifi, though it probably should work fine.

Not sure how to proxy interfaces with lwip / micropython but I'm guessing there will probably be some standard patterns for this.

On most ports, features are generally enabled like:

make -j BOARD=RPI_PICO2_W CFLAGS_EXTRA="-DMICROPY_HW_NETWORK_USBNET=1"

That plug gap alignment error is a strange one, I actually got it yesterday on a RPI build on an unrelated branch. I think it's related to the compilation of picotool more than the firmware issue, don't know where it came.

I haven't used this change on a board with wifi, though it probably should work fine.

Not sure how to proxy interfaces with lwip / micropython but I'm guessing there will probably be some standard patterns for this.

Thanks a lot, I'm new to Python and really stumbled upon this out of sheer luck!

EDIT: It seems to be built but not linked, none of the ./firmware.* files contain the string USBNET.

NOTE:
I did make clean, then cmake -DMICROPY_BOARD=RPI_PICO2_W -DMICROPY_HW_NETWORK_USBNET=1 and then finally make -j BOARD=RPI_PICO2_W CFLAGS_EXTRA="-DMICROPY_HW_NETWORK_USBNET=1"

I hope this makes it into master as a default built option

Thanks for your testing information, I'll have to look into making it easier to include on-demand with the make command.

At the moment I'm not expecting it to be built in by default as it adds a reasonable amount of flash/ram use; I've got a build variant of the original Pico added as an example but that's not really helping with your board.

SOLVED: I ported the commit to the RPI_PICO2_W, it works.

I'm not sure how one should forward an interface to another (but that's unrelated to this PR)

Great to hear @xplshn
I finally figured out how to pass defines on the command line during make, it's different to the other ports:

make -j BOARD=RPI_PICO2_W clean

CFLAGS="-DMICROPY_HW_NETWORK_USBNET=1" make  -j BOARD=RPI_PICO2_W

@andrewleech do you think an example of RNDIS usage could be provided? I want to finish my wifi2usb.py program, I'd appreciate help, perhaps this could become the RNDIS usage example :)

import network
import rp2
import time
import sys
import machine
from machine import Pin
import select
import socket

pico_led = Pin("LED", Pin.OUT)

SCAN_TIMEOUT = 5000

class WiFi2USB:
    def __init__(self):
        self.usb_net = network.USB_NET()
        self.usb_net.active(True)

        self.wlan = network.WLAN(network.STA_IF)
        self.wlan.active(True)

        self.networks = []
        self.current_ssid = ""
        self.current_password = ""
        self.connected = False

    def scan_networks(self):
        """Scan for available WiFi networks"""
        pico_led.on()
        print("Scanning for networks...")
        self.networks = self.wlan.scan()
        pico_led.off()
        return self.networks

    def connect_wifi(self, ssid, password):
        """Connect to a WiFi network with given SSID and password"""
        pico_led.off()
        print(f"Connecting to {ssid}...")

        self.wlan.connect(ssid, password)

        start_time = time.ticks_ms()
        while not self.wlan.isconnected():
            # Check for cancel with BOOTSEL button
            if rp2.bootsel_button() == 1:
                print("\nConnection attempt cancelled")
                return False

            if time.ticks_diff(time.ticks_ms(), start_time) > SCAN_TIMEOUT:
                print("\nConnection timeout - please try again")
                return False

            pico_led.on()
            time.sleep(0.1)
            pico_led.off()
            time.sleep(0.1)

        ip = self.wlan.ifconfig()[0]
        print(f'Connected on {ip}')
        pico_led.on()
        self.connected = True
        self.current_ssid = ssid
        self.current_password = password
        return True

    def disconnect_wifi(self):
        """Disconnect from WiFi network"""
        if self.connected:
            self.wlan.disconnect()
            self.connected = False
            pico_led.off()
            print("Disconnected from WiFi")

    def get_network_info(self):
        """Return current network configuration"""
        if self.connected:
            return {
                'ssid': self.current_ssid,
                'ip': self.wlan.ifconfig()[0],
                'subnet': self.wlan.ifconfig()[1],
                'gateway': self.wlan.ifconfig()[2],
                'dns': self.wlan.ifconfig()[3],
                'status': 'Connected'
            }
        else:
            return {'status': 'Not connected'}

    def test_connectivity(self):
        """Test network connectivity"""
        if not self.connected:
            print("Not connected to any network")
            return False

        try:
            ip = self.wlan.ifconfig()[0]
            print(f"Connected with IP: {ip}")
            print("Gateway: " + self.wlan.ifconfig()[2])

            # TODO: ping a reliable site here
            print("Connectivity test passed")
            return True
        except:
            print("Connectivity test failed")
            return False

    def get_usb_status(self):
        """Return USB RNDIS status"""
        return {
            'active': self.usb_net.active(),
            'ifconfig': self.usb_net.ifconfig()
        }

    def forward_traffic(self):
        """Forward traffic between WiFi and USB interfaces"""
        try:
            wlan_sock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)
            usb_sock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_IP)

            wlan_sock.bind((self.wlan.ifconfig()[0], 0))
            usb_sock.bind((self.usb_net.ifconfig()[0], 0))

            poller = select.poll()
            poller.register(wlan_sock, select.POLLIN)
            poller.register(usb_sock, select.POLLIN)

            print("Forwarding traffic between WiFi and USB interfaces...")

            while True:
                events = poller.poll(1000)
                for fd, event in events:
                    if fd == wlan_sock.fileno():
                        data, addr = wlan_sock.recvfrom(1500)
                        usb_sock.sendto(data, (self.usb_net.ifconfig()[0], 0))
                    elif fd == usb_sock.fileno():
                        data, addr = usb_sock.recvfrom(1500)
                        wlan_sock.sendto(data, (self.wlan.ifconfig()[0], 0))
        except OSError as e:
            print(f"Socket error: {e}")
        except Exception as e:
            print(f"Unexpected error: {e}")

class CLI:
    def __init__(self, wifi2usb):
        self.wifi2usb = wifi2usb
        self.current_menu = self.main_menu

    def start(self):
        """Start the CLI interface"""
        print("\n" + "=" * 50)
        print("WiFi2USB Dongle - CLI Interface")
        print("=" * 50)

        while True:
            try:
                self.current_menu()
            except KeyboardInterrupt:
                print("\nExiting CLI Interface")
                break

    def main_menu(self):
        """Display the main menu"""
        print("\nMAIN MENU:")
        print("1. Reconfigure network")
        print("2. See network config")
        print("3. Test network connectivity")
        print("4. Enter microPython REPL")
        print("0. Exit")

        choice = input("\nEnter your choice: ")

        if choice == "1":
            self.current_menu = self.network_selection_menu
        elif choice == "2":
            self.network_config_menu()
        elif choice == "3":
            self.connectivity_test_menu()
        elif choice == "4":
            self.enter_repl()
        elif choice == "0":
            print("Exiting...")
            sys.exit()
        else:
            print("Invalid choice, please try again")

    def network_selection_menu(self):
        """Display network selection menu"""
        print("\nScanning for networks...")
        networks = self.wifi2usb.scan_networks()

        print("\nAvailable Networks:")
        print("   Network Name                [#Channel] |Other info|")
        print("-" * 60)

        for i, network in enumerate(networks, 1):
            try:
                ssid = network[0].decode('utf-8')
                bssid = network[1]  # MAC address in bytes
                channel = network[2]
                rssi = network[3]
                authmode = network[4]

                # Format MAC address
                mac = ':'.join('{:02x}'.format(b) for b in bssid)

                # Safe access to security types with fallback
                security_types = ["Open", "WEP", "WPA-PSK", "WPA2-PSK", "WPA/WPA2-PSK", "WPA3"]
                if 0 <= authmode < len(security_types):
                    security = security_types[authmode]
                else:
                    security = f"Unknown({authmode})"

                print(f"{i}. {ssid:<25} [#{channel}]     |mac:{mac}, db:{rssi}, {security}|")
            except Exception as e:
                print(f"{i}. [Error reading network info: {e}]")

        print("-" * 60)
        print("0: go back to last menu")

        try:
            choice = input("\nSelect network: ")

            if choice == "0":
                self.current_menu = self.main_menu
                return

            idx = int(choice) - 1
            if 0 <= idx < len(networks):
                selected_ssid = networks[idx][0].decode('utf-8')
                password = input(f"Enter password for {selected_ssid}: ")

                if self.wifi2usb.connect_wifi(selected_ssid, password):
                    print(f"Successfully connected to {selected_ssid}")
                    self.wifi2usb.forward_traffic()
                else:
                    print(f"Failed to connect to {selected_ssid}")

                # Return to main menu after connection attempt
                self.current_menu = self.main_menu
            else:
                print("Invalid selection")
        except ValueError:
            print("Please enter a valid number")

    def network_config_menu(self):
        """Display current network configuration"""
        print("\nCurrent Network Configuration:")
        print("-" * 50)

        # WiFi status
        wifi_info = self.wifi2usb.get_network_info()
        print("WiFi Status:", wifi_info['status'])

        if wifi_info['status'] == 'Connected':
            print(f"SSID: {wifi_info['ssid']}")
            print(f"IP Address: {wifi_info['ip']}")
            print(f"Subnet Mask: {wifi_info['subnet']}")
            print(f"Gateway: {wifi_info['gateway']}")
            print(f"DNS Server: {wifi_info['dns']}")

        # USB RNDIS status
        usb_status = self.wifi2usb.get_usb_status()
        print("\nUSB Network Interface:")
        print(f"Active: {usb_status['active']}")
        print(f"IP Configuration: {usb_status['ifconfig']}")

        print("-" * 50)
        input("Press Enter to return to main menu...")

    def connectivity_test_menu(self):
        """Test network connectivity"""
        print("\nTesting Network Connectivity:")
        print("-" * 50)

        self.wifi2usb.test_connectivity()

        print("-" * 50)
        input("Press Enter to return to main menu...")

    def enter_repl(self):
        """Enter MicroPython REPL"""
        print("\nEntering MicroPython REPL...")
        print('Type "exit()" to return to CLI')
        print("-" * 50)

        # TODO: Replace this, it sucks
        while True:
            try:
                code = input(">>> ")
                if code.strip() == "exit()":
                    break
                try:
                    result = eval(code)
                    if result is not None:
                        print(result)
                except SyntaxError:
                    exec(code)
            except Exception as e:
                print(f"Error: {e}")

        print("-" * 50)
        print("Returned from REPL")

def main():
    wifi2usb = WiFi2USB()
    cli = CLI(wifi2usb)
    cli.start()

if __name__ == "__main__":
    main()

@xplshn I really don't know anything about how packet forwarding is supposed to work. Does your application there seem to work at all?
For your repl bit it would probably be better to integrate aiorepl from micropython-lib

FWIW I've got an example usage of this published at https://github.com/andrewleech/mpy-usb-net-mqtt-demo/

ps. it's not RNDIS, that's the older standard protocol for usb network adapters. NCM is the newer one, it's a different protocol / driver behind the scenes. They should basically work the same as far as users are concerned though.

@xplshn I really don't know anything about how packet forwarding is supposed to work. Does your application there seem to work at all?

My code works, it stablishes a WLAN connection, and an NCM connection (as you said), but packet forwarding isn't handled at all, because I do not know how, if someone can shed some light, I'd appreciate it

For your repl bit it would probably be better to integrate aiorepl from micropython-lib

FWIW I've got an example usage of this published at https://github.com/andrewleech/mpy-usb-net-mqtt-demo/

Thank you! didn't know about these two

Was pointed at this PR regarding a possible solution for an ESP32 project I'm working on. Having problems compiling machine_usb_device.c in extmod due to missing code for 'mp_usbd_enable_class_cdc', 'mp_usbd_enable_class_msc' and 'mp_usbd_enable_class_ncm' and the define for MICROPY_HW_USB_MSC is in shared code but not in all ports.

Happy to contribute/test for ESP32 if interested.

I would love to see this PR get included. I rebased it on current master and it does seem to compile correctly for STM32 (tried BOARD=NUCLEO_WB55) - but I have no board for testing it with. My very basic test:

export BOARD=NUCLEO_WB55
export MICROPY_PY_LWIP=1
export CFLAGS_EXTRA="-DMICROPY_HW_TINYUSB_STACK=1 -DMICROPY_HW_NETWORK_USBNET=1"
make -C mpy-cross
cd ports/stm32
make -j clean
make -j submodules
make -j

goes well.

I then started fiddling a bit with ports/esp32 - here things get messy. Expressif already includes a custom TinyUSB implementation, that sports an NCM driver, under managed_components/espressif__tinyusb/. Moreover, ESP32 does not make use of MICROPY_PY_LWIP - and maybe even conflicts with it.

Basic test for a generic ESP32S3 board:

export BOARD=ESP32_GENERIC_S3
export CFLAGS_EXTRA="-DMICROPY_HW_NETWORK_USBNET=1"
make -C mpy-cross
cd ports/esp32
make -j clean
make -j submodules
make -j

@andrewleech - can I provide some help on pushing this PR further?

The current direction I've been pursuing is trying to make this runtime configurable such that ncm can be enabled/disabled in boot.py like is common with other network interfaces.

My initial attempts at this have been larger than expected (in terms of flash size) so need further work.

That being said, I think that probably shouldn't have to be a blocking feature for this PR, I've already been able to work on that in isolation to this branch.

I think there are still other changes in this branch though that will need to be split out into their own branch / PR - I'll have to have another go getting it cleaned up soon.

Thanks a lot for your continued effort in this topic - it will surely contribute to a much improved USB support for MP.

If you think I could help in any way to make it happen, please ask.

I managed to make ports/esp32 compile with MICROPY_HW_NETWORK_USBNET=1. The interface is not working, but I can see it's announcing itself as NCM to my Linux laptop - so I got one step further.

I lost an insane amount of time because configuring Espressif TinyUSB does not work with the same setup used by MicroPython CMake: namely, MP uses CFLAGS_EXTRA while idf.py requires EXTRA_CFLAGS. So the env needs something like:

BOARD=ESP32_GENERIC_S3
BOARD_VARIANT=SPIRAM_OCT
CFLAGS_EXTRA="-DMICROPY_HW_NETWORK_USBNET=1"
export BOARD BOARD_VARIANT CFLAGS_EXTRA
# idf.py uses a different env var!!!
export EXTRA_CFLAGS="$CFLAGS_EXTRA"

My branch is here.

This has been rebased and pretty significantly reworked. The STM32 TinyUSB enablement and a couple of standalone fixes have been split out into separate PRs (#19070, #19071) so this one now focuses on NCM networking and the runtime class selection API.

The big changes from the previous version:

The "per-driver runtime USB class control" approach with the opaque C type and internal flags byte is gone. Instead there are simple integer flag constants (BUILTIN_FLAG_CDC, BUILTIN_FLAG_MSC, BUILTIN_FLAG_NCM) that get OR'd together and assigned to builtin_driver in boot.py. This builds a dynamic configuration descriptor at runtime with compact interface/endpoint numbering. Much simpler at both the Python and C level.

BUILTIN_DEFAULT is now CDC-only. MSC and NCM are opt-in via the flag constants. Without RUNTIME_DEVICE (ie. boards that don't have machine.USBDevice) all compiled-in classes are exposed by default in the static descriptor.

The NCM netif is now auto-initialised at boot from mod_network_init() rather than waiting for Python code to create a network.USB_NET() object. This is necessary because TinyUSB's NCM driver (in v0.19+) starts receiving as soon as the host sends SET_INTERFACE during enumeration, which happens before any Python code runs. Without the netif being ready at that point, the receive path stalls.

The DHCP server no longer advertises a default gateway on the NCM link. The previous version caused the host to route all internet traffic over USB which broke outbound connectivity. There's a new send_router flag on the shared dhcpserver that controls this.

Port Makefiles for alif, mimxrt, renesas-ra and samd now use TinyUSB's upstream tinyusb.mk instead of hand-maintained file lists. NCM is auto-enabled on any port with lwIP and TinyUSB via MICROPY_HW_NETWORK_USBNET.

Tested on NUCLEO_WB55 with both RUNTIME_DEVICE=1 and RUNTIME_DEVICE=0, ping working at ~1.8ms RTT over the NCM link. Haven't re-tested on rp2, mimxrt, renesas-ra or samd yet (build-tested only).

The branch has three phases that could potentially be split into separate stacked PRs if that would make review easier: background network support prerequisites, the NCM driver itself, and the runtime USB class selection API.

Octoprobe PR report

• Duration: 2h 12min, 88741 tests run, 90ms/test
• Tested ports: esp32 esp8266 mimxrt nrf rp2 samd stm32
• Summary Report , Logdirectory