WIP: Python/PyPI packaging for Unix port#19425
Conversation
|
Code size report: |
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| matrix: | ||
| os: [ubuntu-latest, macos-latest] |
There was a problem hiding this comment.
Unix port does not work on Windows, I think? At least there were some compile errors when I had it enabled - I just skipped it to focus on the other platforms first.
There was a problem hiding this comment.
But then perhaps not use the _unix suffix in the package name as that prohibits future platforms
There was a problem hiding this comment.
@Josverl I am open for either! I don't really know Windows port much. But if it is sufficiently similar conceptually then a "micropython" package could make sense.
| return ret; | ||
| } | ||
|
|
||
| PYBIND11_MODULE(micropython_run, m) { |
There was a problem hiding this comment.
Using pybind11 for a wrapper to call an executable seems quite heavy. Why not just use the subprocess module to call it?
There was a problem hiding this comment.
The specific reason is for Pyodide, the packaging target that allows this in browser - specifically for Jupyter Lite (Jupyter Lab in browser). That environment does not have processes, so the MicroPython interpreter must execute "in process". That is a key feature for me - since I want to teach machine learning for MicroPython, where participants need both CPython (for training ML models) and MicroPython (for the on-device code) - and Jupyter Lite will enable a zero-install experience for that.
I have done a succesful proof-of-concept for that before. Will add that here also (can be in this MR or as a follow-up). Just requires a bit more fiddling with the build configuration and Emscripten toolchain, so I did not want to block the initial RFC on it :)
There was a problem hiding this comment.
The module-approach also would make possible a proper API for the MicroPython interpreter.
vm = micropython_unix.Interpreter()
vm.execute("a = 1; b=2; c=a+b")
c = vm.locals['c']
This is not an important usecase for me*, but might be of interest to others? Out-of-scope for now though!
*In combined CPython+MicroPython ML/DS dataflows I use scripts in MicroPython that take standardized files in and out, and then a wrapper in CPython that handles serialization etc. Which one can also use for on-device tests, and serialization overhead is not an issue.
There was a problem hiding this comment.
The module-approach also would make possible a proper API for the MicroPython interpreter.
With the right set of compilation flags and if it can be packaged as a shared library, this could - in theory - be used to replace most of mpy-cross C code via ctypes :)
There was a problem hiding this comment.
The specific reason is for Pyodide, the packaging target that allows this in browser
Would you not want to use the javascript port of MicroPython in this case?
Allows it to be used by other programs/modules, such as a CPython module for running code
emscripten fails if this is present
Builds a Python package that provides a "micropython" script. Can then be easily distributed on PyPi.
8db5831 to
a197a8e
Compare
|
Thanks a lot for review feedback @dlech and @agatti ! I did a pass today and was able to resolve multiple of them. I got the packages working with musllinux (Alpine). And I got it working in browser - Pyodide with Jupyter Lite. So now one can go to https://jupyter.org/try-jupyter/lab/ and do like the following to run MicroPython code: |
Tested:
|
Summary
Builds a Python package for Unix port, and sets up MicroPython.
This is an early version. High-level feedback and review comments are very welcomed. Once the overall approach is agreed on, I will clean it up accordingly.
Suggested next steps:
Prebuilt packages are available via
pip install jonnor-micropython-unix. For Mac OS (ARM) and Linux (x86_64). It should work on Windows Subsystem for Linux using the Linux packages. I chose to support CPython 3.10+ (2021). I believe this is a practical starting point in terms of coverage, that should cover the majority of users.It is possible to build and install the package from source by doing
pip install "git+https://github.com/jonnor/micropython.git@unix-python-package-1#subdirectory=ports/unix". This requires build tools such asmakeand a compiler.Testing
Tested installing the built package a couple of different Linux distributions.
To be tested: Windows Subsystem for Linux, Mac OS
Generative AI
I used generative AI tools when creating this PR, but a human has checked the
code and is responsible for the code and the description above.
Details: I did not use any code as-is from gen AI. It was primarily used for searching for information online, and for trying to understand some compile error messages.