Current status

TODO

Filter host devices to only include devices which may be assigned (check FLR capability, whether behind bridge, non-page-aligned MMIO BARs)

Before assignment, reset the device, unbind its driver and bind pci-stub.ko

Completed

Support in kvm.ko and qemu-kvm

pci-stub.ko to reserve host devices

libvirt host device enumeration and assignment

libvirt host device assignment

Detailed Description

KVM guests usually have access to either virtio devices or emulated devices. If the guest has access to suitable drivers, then virtio is preferred because it allows high performance to be achieved.

On host machines which have Intel VT-d or AMD IOMMU hardware support, another option is possible. PCI devices may be assigned directly to the guest, allowing the device to be used with minimal performance overhead.

However, device assignment is not always the best option even when it is available. Problems include:

All of the guest's memory must kept permanently in memory. This is because the guest may program the device with any address in its address space and the hypervisor has no way of handling a DMA page fault.

It isn't possible to migrate the guest to another host. Even if the exact same hardware exists on the remote host, it is impossible for the hypervisor to migrate the device state between hosts.

Graphics cards cannot currently be assigned because they require access to the video BIOS.

In summary, PCI device assignment is possible given the appropriate hardware, but it is only suitable in certain situations where the flexibility of memory over-commit and migration is not required.

Benefit to Fedora

Fedora users will be able to assign PCI network cards, hard disk controllers, phone line termination cards etc. to their virtual machines.

Device assignment is an important feature for any virtualization platform. As such, the feature will improve Fedora's virtualization standings in any competitive analysis.

Scope

The core device assignment support in the kernel includes:

IOMMU API

VT-d and AMD IOMMU support

Device assignment in kvm.ko

Support is also required in qemu-kvm, the userspace component of KVM.

libvirt, python-virtinst and virt-manager also require support to be added in order to allow users to easily assign devices. Not only is the ability to assign devices needed, but also the ability to list what devices are available to assign.

Further complications include:

Preventing a guest assigned device from being used in the host

Detecting which host devices are assignable

Ensuring that devices are properly reset.

How To Test

Perhaps the most straightforward test is to use virt-manager create a Fedora 11 KVM guest with a PCI network card.

Run virt-manager, click on New and go through the usual process of creating a guest

In the network configuration screen, choose the "assign physical device" option and select an available network card from the list

Start the guest install process, check that it completes successfully

Check the newly installed guest reboots and has access to the assigned network card

Notes:

You need to have a machine with Intel VT-d or AMD IOMMU support.

If you have only one network card on the host machine, the host will not be able to access the network during the test. So, e.g. you would need to local install media to start the installation.

User Experience

The user experience is similar to that described above. Users will be able to easily assign PCI devices to their KVM guests.

Dependencies

Outside of the changes described above for the kernel, kvm, libvirt, python-virtinst and virt-manager, the only other changes which should be needed is to fix any devices drivers which fail to reset devices correctly and leave devices in a broken state.

Contingency Plan

If the tools support is not complete in time, no contingency is needed. The funcionality just wouldn't be available.

However, if the functionality was so broken that we didn't want to expose it to users, we might have to disable the support in the tools - e.g. remove the device assignment UI from virt-manager.