Skip to content

Conversation

@DemiMarie
Copy link
Contributor

This adds support for a generic vhost-user-generic device. All information about this device must be provided to Cloud Hypervisor via the command-line or API.

Fixes #7189

@DemiMarie DemiMarie requested a review from a team as a code owner July 25, 2025 02:22
@DemiMarie DemiMarie force-pushed the generic-vhost-user branch 6 times, most recently from aed497d to cd6b328 Compare July 26, 2025 02:33
@DemiMarie DemiMarie marked this pull request as draft July 28, 2025 19:35
@DemiMarie DemiMarie force-pushed the generic-vhost-user branch 4 times, most recently from b101b21 to 64b241c Compare August 8, 2025 01:13
@DemiMarie DemiMarie force-pushed the generic-vhost-user branch 2 times, most recently from 2b46ed3 to 5c8e3f7 Compare August 9, 2025 17:54
@rbradford
Copy link
Member

Can you give an example of what a vhost-user-generic device is?

@DemiMarie DemiMarie marked this pull request as ready for review August 11, 2025 04:26
@DemiMarie
Copy link
Contributor Author

A vhost-user-generic device is a vhost-user device that is implemented entirely by the backend. All parameters, such as the maximum number of queues and the features, must be passed to Cloud Hypervisor via API or the command line. This allows using vhost-user devices that Cloud Hypervisor does not know about, such as sound or (in the future) GPU. It is also possible for other devices to wrap the vhost-user-generic implementation if they do not need any specific handling.

@liuw
Copy link
Member

liuw commented Aug 14, 2025

The QEMU patch referenced in #7189 -- https://lore.kernel.org/qemu-devel/[email protected]/ -- shows that it is still in RFC status. What's the latest status in QEMU?

There is a flag VHOST_USER_PROTOCOL_F_PROBE defined in the QEMU patch, but that's not accepted. Your patches don't use that flag.

How will this work in practice? Do you have a working backend to support this already?

@DemiMarie
Copy link
Contributor Author

DemiMarie commented Aug 15, 2025

The QEMU patch referenced in #7189 -- https://lore.kernel.org/qemu-devel/[email protected]/ -- shows that it is still in RFC status. What's the latest status in QEMU?

I don’t think the patch was accepted. QEMU does have a generic vhost-user device, but (like this patch) it requires parameters to be passed by the user rather than fetching them from the backend. QEMU also requires a trivial source-code patch to enable the feature, which this patch doesn’t need.

There is a flag VHOST_USER_PROTOCOL_F_PROBE defined in the QEMU patch, but that's not accepted. Your patches don't use that flag.

You are correct. VHOST_USER_PROTOCOL_F_PROBE is needed to automatically obtain various information from the backend. This patch requires the information to be provided at device creation time instead.

How will this work in practice? Do you have a working backend to support this already?

This works with an unmodified virtiofsd, but the caller must explicitly provide all of the parameters needed. If I did not make any copy-and-paste mistakes, I used the following option to make a virtio-fs device:

--vhost-user-generic \
"socket=build/virtiofsd.sock,virtio_id=26,avail_features=$((0x5970000000)),avail_protocol_features=$((0x920b)),queue_size=1024,num_queues=1,min_queues=1,id=virtiofs0"

This interface is deliberately low-level to allow maximum flexibility. Callers are expected to know the parameters for their particular device. Higher-level tooling is expected to provide a user-friendly interface.

@alyssais
Copy link
Member

Needs some clippy fixes.

@DemiMarie DemiMarie force-pushed the generic-vhost-user branch 2 times, most recently from 595b465 to 986efa7 Compare November 22, 2025 02:58
let num_queues = parser
.convert("num_queues")
.map_err(Error::ParseVhostUser)?
.ok_or(Error::ParseVhostUserNumQueuesMissing)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would be better to take queue_sizes, and infer num_queues from that? That would allow specifying queues of different types — e.g. crosvm GPU uses one queue of size 256, and another of size 16.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide some more details here? I’m not sure how the API for this should look.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--user-vhost-device queue_sizes=[256,16], and then you know there are two queues.

@DemiMarie DemiMarie force-pushed the generic-vhost-user branch 6 times, most recently from 7d529a8 to 301b753 Compare December 7, 2025 21:35
@DemiMarie DemiMarie force-pushed the generic-vhost-user branch 4 times, most recently from f7667e5 to 5ee5d30 Compare December 20, 2025 04:48
@DemiMarie DemiMarie requested a review from alyssais December 20, 2025 06:38
// Copyright 2019 Intel Corporation. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0

use std::io::Write as _;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this as _ is idiomatic. We don't do it anywhere else that I can see.

VIRTIO_F_VERSION_1, VirtioCommon, VirtioDevice, VirtioInterrupt, VirtioSharedMemoryList,
};

const DEFAULT_QUEUE_NUMBER: usize = 2;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we even have a default?

#[repr(C, packed)]
pub struct VirtioVhostUserConfig {
pub num_request_queues: u32,
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still relevant.

pub num_request_queues: u32,
}

// SAFETY: only a series of integers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// SAFETY: only a series of integers
// SAFETY: only an integer

10,
&IntegerList(vec![u16::MAX.into(), 20u16.into()]),
virtio_devices::vhost_user::DEFAULT_VIRTIO_FEATURES,
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd rather see construction and checking of these inlined. As it is it's quite difficult to see what's actually being tested for each case.


/// Cannot create virtio-vhost-user device
#[error("Cannot create virtio-vhost-user device")]
CreateVirtioVhostUser(#[source] virtio_devices::vhost_user::Error),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this supposed to be renamed?

pub struct PvmemcontrolConfig {}

#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct AvailProtocolFeatures(pub VhostUserProtocolFeatures);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this being used?


pub fn default_vhost_user_config_num_queues() -> usize {
1
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should have been deleted.

@DemiMarie DemiMarie force-pushed the generic-vhost-user branch 4 times, most recently from 0cdc95c to c52ba0a Compare December 31, 2025 16:39
DemiMarie added a commit to DemiMarie/cloud-hypervisor that referenced this pull request Dec 31, 2025
This was breaking CI for PR cloud-hypervisor#7221.

Signed-off-by: Demi Marie Obenour <[email protected]>
DemiMarie added a commit to DemiMarie/cloud-hypervisor that referenced this pull request Dec 31, 2025
This breaks CI for cloud-hypervisor#7221.

Signed-off-by: Demi Marie Obenour <[email protected]>
This breaks CI for cloud-hypervisor#7221.

Signed-off-by: Demi Marie Obenour <[email protected]>
This adds support for a generic vhost-user-generic device.  All
information about this device must be provided to Cloud Hypervisor via
the command-line or API.

Fixes cloud-hypervisor#7189

Signed-off-by: Demi Marie Obenour <[email protected]>
DemiMarie added a commit to DemiMarie/cloud-hypervisor that referenced this pull request Dec 31, 2025
This breaks CI for cloud-hypervisor#7221.

Signed-off-by: Demi Marie Obenour <[email protected]>
DemiMarie added a commit to DemiMarie/cloud-hypervisor that referenced this pull request Dec 31, 2025
This breaks CI for cloud-hypervisor#7221.

Signed-off-by: Demi Marie Obenour <[email protected]>
DemiMarie added a commit to DemiMarie/cloud-hypervisor that referenced this pull request Dec 31, 2025
This breaks CI for cloud-hypervisor#7221.

Signed-off-by: Demi Marie Obenour <[email protected]>
DemiMarie added a commit to DemiMarie/cloud-hypervisor that referenced this pull request Jan 1, 2026
This breaks CI for cloud-hypervisor#7221.

Signed-off-by: Demi Marie Obenour <[email protected]>
DemiMarie added a commit to DemiMarie/cloud-hypervisor that referenced this pull request Jan 1, 2026
This breaks CI for cloud-hypervisor#7221.

Signed-off-by: Demi Marie Obenour <[email protected]>
DemiMarie added a commit to DemiMarie/cloud-hypervisor that referenced this pull request Jan 1, 2026
This breaks CI for cloud-hypervisor#7221.

Signed-off-by: Demi Marie Obenour <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for generic vhost-user devices

4 participants