We’d like to announce the availability of the QEMU 9.1.0 release. This release contains 2800+ commits from 263 authors.
You can grab the tarball from our download page. The full list of changes are available in the changelog.
Highlights include:
Thank you to everybody who contributed to this release, whether that was by writing code, reporting bugs, improving documentation, testing, or providing the project with CI resources. We couldn’t do these without you!
]]>The KVM Forum 2024 conference will take place in Brno, Czech Republic on September 22-23, 2024. KVM Forum brings together the Linux virtualization community, especially around the KVM stack, including QEMU and other virtual machine monitors.
The Call for Presentations is open until June 8, 2024. You are invited to submit presentation proposals via the KVM Forum CfP page. All presentation slots will be 25 minutes + 5 minutes for questions.
Suggested topics include:
We’d like to announce the availability of the QEMU 9.0.0 release. This release contains 2700+ commits from 220 authors.
You can grab the tarball from our download page. The full list of changes are available in the changelog.
Highlights include:
Thank you to everybody who contributed to this release, whether that was by writing code, reporting bugs, improving documentation, testing, or providing the project with CI resources. We couldn’t do these without you!
]]>We’d like to announce the availability of the QEMU 8.2.0 release. This release contains 3200+ commits from 238 authors.
You can grab the tarball from our download page. The full list of changes are available in the changelog.
Highlights include:
Thank you to everybody who contributed to this release, whether that was by writing code, reporting bugs, improving documentation, testing, or providing the project with CI resources. We couldn’t do these without you!
]]>We’d like to announce the availability of the QEMU 8.1.0 release. This release contains 2900+ commits from 250 authors.
You can grab the tarball from our download page. The full list of changes are available in the changelog.
Highlights include:
Thank you to everybody who contributed to this release, whether that was by writing code, reporting bugs, improving documentation, testing, or providing the project with CI resources. We couldn’t do these without you!
]]>We’d like to announce the availability of the QEMU 8.0.0 release. This release contains 2800+ commits from 238 authors.
You can grab the tarball from our download page. The full list of changes are available in the changelog.
Highlights include:
Thank you to everybody who contributed to this release, whether that was by writing code, reporting bugs, improving documentation, testing, or providing the project with CI resources. We couldn’t do these without you!
]]>Building QEMU is a complex task, split across several programs.
the configure script finds the host and cross compilers that are needed
to build emulators and firmware; Meson prepares the build environment
for the emulators; finally, Make and Ninja actually perform the build,
and in some cases they run tests as well.
In addition to compiling C code, many build steps run tools and scripts which are mostly written in the Python language. These include processing the emulator configuration, code generators for tracepoints and QAPI, extensions for the Sphinx documentation tool, and the Avocado testing framework. The Meson build system itself is written in Python, too.
Some of these tools are run through the python3 executable, while others
are invoked directly as sphinx-build or meson, and this can create
inconsistencies. For example, QEMU’s configure script checks for a
minimum version of Python and rejects too-old interpreters. However,
what would happen if code run by Sphinx used a different version?
This situation has been largely hypothetical until recently; QEMU’s
Python code is already tested with a wide range of versions of the
interpreter, and it would not be a huge issue if Sphinx used a different
version of Python as long as both of them were supported. This will
change in version 8.1 of QEMU, which will bump the minimum supported
version of Python from 3.6 to 3.8. While all the distros that QEMU
supports have a recent-enough interpreter, the default on RHEL8 and
SLES15 is still version 3.6, and that is what all binaries in /usr/bin
use unconditionally.
As of QEMU 8.0, even if configure is told to use /usr/bin/python3.8
for the build, QEMU’s custom Sphinx extensions would still run under
Python 3.6. configure does separately check that Sphinx is executing
with a new enough Python version, but it would be nice if there were
a more generic way to prepare a consistent Python environment.
This post will explain how QEMU 8.1 will ensure that a single interpreter is used for the whole of the build process. Getting there will require some familiarity with Python packaging, so let’s start with virtual environments.
It is surprisingly hard to find what Python interpreter a given script
will use. You can try to parse the first line of the script, which will
be something like #! /usr/bin/python3, but there is no guarantee of
success. For example, on some version of Homebrew /usr/bin/meson
will be a wrapper script like:
#!/bin/bash
PYTHONPATH="/usr/local/Cellar/meson/0.55.0/lib/python3.8/site-packages" \
exec "/usr/local/Cellar/meson/0.55.0/libexec/bin/meson" "$@"
The file with the Python shebang line will be hidden somewhere in
/usr/local/Cellar. Therefore, performing some kind of check on the
files in /usr/bin is ruled out. QEMU needs to set up a consistent
environment on its own.
If a user who is building QEMU wanted to do so, the simplest way would
be to use Python virtual environments. A virtual environment takes an
existing Python installation but gives it a local set of Python packages.
It also has its own bin directory; place it at the beginning of your
PATH and you will be able to control the Python interpreter for scripts
that begin with #! /usr/bin/env python3.
Furthermore, when packages are installed into the virtual environment
with pip, they always refer to the Python interpreter that was used to
create the environment. Virtual environments mostly solve the consistency
problem at the cost of an extra pip install step to put QEMU’s build
dependencies into the environment.
Unfortunately, this extra step has a substantial downside. Even though
the virtual environment can optionally refer to the base installation’s
installed packages, pip will always install packages from scratch
into the virtual environment. For all Linux distributions except RHEL8
and SLES15 this is unnecessary, and users would be happy to build QEMU
using the versions of Meson and Sphinx included in the distribution.
Even worse, pip install will access the Python package index (PyPI)
over the Internet, which is often impossible on build machines that
are sealed from the outside world. Automated installation of PyPI
dependencies may actually be a welcome feature, but it must also remain
strictly optional.
In other words, the ideal solution would use a non-isolated virtual
environment, to be able to use system packages provided by Linux
distributions; but it would also ensure that scripts (sphinx-build,
meson, avocado) are placed into bin just like pip install does.
When it comes to packages, Python surely makes an effort to be confusing.
The fundamental unit for importing code into a Python program is called
a package; for example os and sys are two examples of a package.
However, a program or library that is distributed on PyPI consists
of many such “import packages”: that’s because while pip is usually
said to be a “package installer” for Python, more precisely it installs
“distribution packages”.
To add to the confusion, the term “distribution package” is often shortened to either “package” or “distribution”. And finally, the metadata of the distribution package remains available even after installation, so “distributions” include things that are already installed (and are not being distributed anywhere).
All this matters because distribution metadata will be the key to
building the perfect virtual environment. If you look at the content
of bin/meson in a virtual environment, after installing the package
with pip, this is what you find:
#!/home/pbonzini/my-venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from mesonbuild.mesonmain import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())
This looks a lot like automatically generated code, and in fact it is;
the only parts that vary are the from mesonbuild.mesonmain import main
import, and the invocation of the main() function on the last line.
pip creates this invocation script based on the setup.cfg file
in Meson’s source code, more specifically based on the following stanza:
[options.entry_points]
console_scripts =
meson = mesonbuild.mesonmain:main
Similar declarations exist in Sphinx, Avocado and so on, and accessing their
content is easy via importlib.metadata (available in Python 3.8+):
$ python3
>>> from importlib.metadata import distribution
>>> distribution('meson').entry_points
[EntryPoint(name='meson', value='mesonbuild.mesonmain:main', group='console_scripts')]
importlib looks up the metadata in the running Python interpreter’s
search path; if Meson is installed under another interpreter’s site-packages
directory, it will not be found:
$ python3.8
>>> from importlib.metadata import distribution
>>> distribution('meson').entry_points
Traceback (most recent call last):
...
importlib.metadata.PackageNotFoundError: meson
So finally we have a plan! configure can build a non-isolated virtual
environment, use importlib to check that the required packages exist
in the base installation, and create scripts in bin that point to the
right Python interpreter. Then, it can optionally use pip install to
install the missing packages.
While this process includes a certain amount of
specialized logic, Python provides a customizable venv
module to create virtual
environments. The custom steps can be performed by subclassing
venv.EnvBuilder.
This will provide the same experience as QEMU 8.0, except that there will
be no need for the --meson and --sphinx-build options to the
configure script. The path to the Python interpreter is enough to
set up all Python programs used during the build.
There is only one thing left to fix…
Remember how we started with a user that creates her own virtual
environment before building QEMU? Well, this would not work
anymore, because virtual environments cannot be nested. As soon
as configure creates its own virtual environment, the packages
installed by the user are not available anymore.
Fortunately, the “appearance” of a nested virtual environment is easy
to emulate. Detecting whether python3 runs in a virtual environment
is as easy as checking sys.prefix != sys.base_prefix; if it is,
we need to retrieve the parent virtual environments site-packages
directory:
>>> import sysconfig
>>> sysconfig.get_path('purelib')
'/home/pbonzini/my-venv/lib/python3.11/site-packages'
and write it to a .pth file in the lib directory of the new virtual
environment. The following demo shows how a distribution package in the
parent virtual environment will be available in the child as well:
A small detail is that configure’s new virtual environment should
mirror the isolation setting of the parent. An isolated venv can be
detected because sys.base_prefix in site.PREFIXES is false.
Right now, QEMU only makes a minimal attempt at ensuring consistency
of the Python environment; Meson is always run using the interpreter
that was passed to the configure script with --python or $PYTHON,
but that’s it. Once the above technique will be implemented in QEMU 8.1,
there will be no difference in the build experience, but configuration
will be easier and a wider set of invalid build environments will
be detected. We will merge these checks before dropping support for
Python 3.6, so that users on older enterprise distributions will have
a smooth transition.
KVM Forum is an annual event that presents a rare opportunity for KVM and QEMU developers and users to discuss the state of Linux virtualization technology and plan for the challenges ahead. Sessions include updates on the state of the KVM virtualization stack, planning for the future, and many opportunities for attendees to collaborate.
This year’s event will be held in Brno, Czech Republic on June 14-15, 2023. It will be in-person only and will be held right before the DevConf.CZ open source community conference.
June 14 will be at least partly dedicated to a hackathon or “day of BoFs”. This will provide time for people to get together and discuss strategic decisions, as well as other topics that are best solved within smaller groups.
We encourage you to submit presentations via the KVM Forum CfP page. Suggested topics include:
The deadline for submitting presentations is April 2, 2023 - 11:59 PM PDT. Accepted speakers will be notified on April 17, 2023.
Admission to KVM Forum and DevConf.CZ is free. However, registration is required and the number of attendees is limited by the space available at the venue.
The DevConf.CZ program will feature technical talks on a variety of topics, including cloud and virtualization infrastructure—so make sure to register for DevConf.CZ as well if you would like to attend.
Both conferences are committed to fostering an open and welcoming environment for everybody. Participants are expected to abide by the Devconf.cz code of conduct and media policy.
]]>QEMU is participating in Google Summer of Code and Outreachy again this year! Google Summer of Code and Outreachy are open source internship programs that offer paid remote work opportunities for contributing to open source. Internships generally run May through August, so if you have time and want to experience open source development, read on to find out how you can apply.
Each intern is paired with one or more mentors, experienced QEMU contributors who support them during the internship. Code developed by the intern is submitted through the same open source development process that all QEMU contributions follow. This gives interns experience with contributing to open source software. Some interns then choose to pursue a career in open source software after completing their internship.
Information on who can apply is here for Google Summer of Code and here for Outreachy. Note that Outreachy initial applications ended on February 6th so only those who have been accepted into Outreachy can apply for QEMU Outreachy internships.
Look through the the list of QEMU project ideas and see if there is something you are interested in working on. Once you have found a project idea you want to apply for, email the mentor for that project idea to ask any questions you may have and discuss the idea further.
You can apply for Google Summer of Code from March 20th to April 4th and apply for Outreachy from March 6th to April 3rd.
Good luck with your applications!
If you have questions about applying for QEMU GSoC or Outreachy, please email Stefan Hajnoczi or ask on the #qemu-gsoc IRC channel.
]]>We’d like to announce the availability of the QEMU 7.2.0 release. This release contains 1800+ commits from 205 authors.
You can grab the tarball from our download page. The full list of changes are available in the Wiki.
Highlights include:
Thank you to everyone involved!
]]>