GPU Passthrough, VGA Passthrough in KVM

To inspire you, I’ve got a video from someone else. Better mutt the volume by the way.
Arch Linux KVM Crysis HD Gpu Passthrough
Or you can download it to see.
Download the video in HD

Here’s the links I refer to:
http://thread.gmane.org/gmane.comp.emulators.kvm.devel/71981
https://bbs.archlinux.org/viewtopic.php?id=162768
https://docs.google.com/document/d/1ef_nfl652L0HLn_wGvnpgjsBJd9LZzaV_-rIcEEoK8Y/edit?pli=1
http://www.linux-kvm.org/page/VGA_device_assignment
http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM

Result:
VGAPassthrough: success in host F19, guest Windows7
GPUPassthrough: success in Fedora-Rawhide

HOST:
CPU: Core i5 3470
GPU: ATI HD Radeon 7850
OS: Fedora-Rawhide
QEMU: qemu-1.5.1
kvm-vgapassthrough
So, here’s the steps

0. Enable the mainboard VxT, iommu and alter the video device to Intel HD

1. See what we have got now.

lspci;lspci -n

We have output below

...
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Pitcairn PRO [Radeon HD 7850]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
...
...
01:00.0 0300: 1002:6819
01:00.1 0403: 1002:aab0
...

You can see the pci bus and vendor.

2. Modify the kernel parameter, morprobe.d and libvirt.conf

Add follow parameters to grub.conf

intel_iommu=on pci-stub.ids=1002:6819,1002:aab0,vfio_iommu_type1.allow_unsafe_interrupts=1

NOTE: If you have got an AMD cpu, please replace “interl_iommu=on” with “iommu=pt iommu=1”
Add modprobe.conf to /etc/modprobe.d/ with this content:

blacklist radeon
options kvm ignore_msrs=1
options kvm allow_unsafe_interrupts=1
options kvm-amd npt=0
options kvm_intel emulate_invalid_guest_state=0
options vfio_iommu_type1 allow_unsafe_interrupts=1

change the following options in /etc/libvirt/qemu.conf:

# The user ID for QEMU processes run by the system instance.
user = "root"

# The group ID for QEMU processes run by the system instance.
group = "root"

......

# If clear_emulator_capabilities is enabled, libvirt will drop all
# privileged capabilities of the QEmu/KVM emulator. This is enabled by
# default.
#
# Warning: Disabling this option means that a compromised guest can
# exploit the privileges and possibly do damage to the host.
#
clear_emulator_capabilities = 0

Reboot.

3. Using scripts below

Version 1: VFIO-Passthrough

File: vfio-bind

#!/bin/bash
modprobe vfio-pci
for var in "$@"; do
        for dev in $(ls /sys/bus/pci/devices/$var/iommu_group/devices); do
                vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
                device=$(cat /sys/bus/pci/devices/$dev/device)
                if [ -e /sys/bus/pci/devices/$dev/driver ]; then
                        echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
                        fi
                echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id
        done
done

Bind the device

./vfio-bind 0000:01:00.0 0000:01:00.1

Start VM

#!/bin/bash
sudo modprobe vfio-pci

sudo qemu-system-x86_64 -no-user-config -nodefaults -m 2048M -smp 4 -boot menu=on \
-net nic -net user -enable-kvm -monitor stdio -vga qxl -global qxl-vga.vram_size=67108864 \
-spice port=6000,ipv4,disable-ticketing \
-device intel-hda,id=sound0,bus=pcie.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
-drive file=Windows7.iso,if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 \
-drive file=/home/lofyer/gpu_passthrough/f17.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pcie.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
-device virtio-balloon-pci,id=balloon0,bus=pcie.0,addr=0x8 \
-M q35 \
-device piix4-ide,bus=pcie.0 \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 \
-fda virtio.vfd

Version 2: PCI-Passthrough

Bind device

#!/bin/bash
modprobe pci-stub
for id in 6819 aab0; do
    echo 1002 $id > /sys/bus/pci/drivers/pci-stub/new_id
done
for pci in 0000:01:00.{0,1}; do
    echo $pci > "/sys/bus/pci/devices/$pci/driver/unbind"
    echo $pci > /sys/bus/pci/drivers/pci-stub/bind
done

Start VM

#!/bin/bash
qemu-system-x86_64 \
-hda ../f17.qcow2 \
-cdrom /run/media/lofyer/Cache/OS_ISO/cn_windows_7_ultimate_with_sp1_x64_dvd_u_677408.iso \
-m 2048 -balloon virtio -smp 4 -enable-kvm \
-device pci-assign,host=01:00.0

24 thoughts on “GPU Passthrough, VGA Passthrough in KVM

  1. does spice work with VM using vfio-pci passthrough and pci-stub passthrough?

    • Spice does not work when the videocard was passed to the vm except spice keyboard and mouse channel. You should use vnc in the VM instead.

  2. Did you patch anything like the thread in arclinux forum said?

    • No.
      I used Fedora Rawhide. Since he said that the patch might have been applied by the time I did this experiment.
      The patches mentioned in that thread was to fix the “reboot issue”, but I skipped this by using dev-eject in the guest before shutting down.

      • 第二个显卡可以实现透传了,但是第一个显卡不能透传,显示器启动没有任何画面,并且停止虚拟机也不能回到linux系统的桌面,请问,如何实现第一个显卡的透传

        • 第一个显卡你意思是你的工作系统用的吗?这个也可以透传,但是传进去后你只能通过ssh上去debug了。

          • pc配置i7 6700,16g内存,两张显卡1060显卡,用的vfio方式,目前的症状是工作用第一张1060显卡,第二张显卡可以成功透传,一旦虚拟机启动,虽然可以成功,但是工作的画面就卡死了,因为显卡是一摸一样的,所以PCI ID号是一样的,但是我如果用CPU集成显卡启动两个虚拟机就没有问题,前提是打了i915vga补丁,前几天打了补丁发现没问题,后来把系统删除重新开始做就发现实现不了,郁闷!还有我用virsh nodedev-detach 解绑第二张显卡,结果发现我工作的第一张显卡也不解绑且不工作了,我在4.x的内核是这样,但是在3.15内核可以任意解绑PCI设备,不知道什么原因

          • 你可以使用subsystem id而非pci id;在关机或重启之前,请把虚拟机里的显卡卸载,可手动可用小程序。

          • 你说的手动,你能给个例子

          • 问题是,为什么3.x的内核可以直接解绑pci显卡,4.x的内核就不行呢,一解绑屏幕就死了

          • 在虚拟机OS内卸载驱动。。

  3. 楼主,
    有问题请教一下,我也实现了穿透了第二块显卡(ATI HD7750)到虚拟机里,用的是第一种方式VFIO。目前显示器联接到HD7750的DVI口可以进到虚拟机桌面,穿透的显卡可用,QEMU窗口是黑的,没有显示输出。我想让虚拟机桌面显示到QEMU窗口或远程桌面工具上而不是直接输出到显卡输出端口。看到你的视频好像虚拟机图像是在Host的窗口中显示的,能否告知怎样详细配置,是直接QEMU窗口显示,还是通过远程桌面工具如VNC Viewer,Spicy连接的?

  4. Interesting content. You should use social websites to increase traffic and make your site go viral.
    There are tools which automate this time consuming process.Visitors can flood your
    website in no time, just search in google for:
    Rixisosa’s Social Automation

  5. 你好!我也在尝试用vfio,pci-stub passthrough pci vga。我想请问一下:
    1 第一次运行qemu,显卡是不是没有输出,必须要装上驱动才可以?
    2 你试过用vfio+ovmf吗?

    • 1. 透传成功之前显卡一直没有输出,都是输出到qemu的monitor,成功后,需要安装显卡驱动,才会输出到物理显卡。
      2. 没有试过 ^_^

  6. 您好,我用了您的Vfio的方法进行passthough,命令是 qemu-system-x86_64 -enable-kvm -M q35 -m 2G -cpu host,kvm=off -smp 6,sockets=1,cores=6,threads=1 -boot d -hda /root/kvm_demo_3.18/win7_64_nvidia_vfio_vnc.img -cdrom /root/kvm_demo/cn_windows_7_ultimate_with_sp1_x64_dvd_u_677408.iso -drive file=/root/kvm_demo/invida_k2000_driver.iso,media=cdrom -device vfio-pci,host=04:00.0,x-vga=on -device vfio-pci,host=04:00.1 -vnc :45 。
    我安装好系统后,装上显卡的驱动,禁用掉qemu提供的标准显卡。然后shutdown guest 系统,重新输入上面的命令,然后用vnc viewer连接,发现是黑屏,上面出现“Guest has not initialized the display(yet)”。 我的显卡(丽台k2000)是“04:00.0”,“04:00.1”是这块显卡上集成的声卡。 我在host上的另一个terminal中输入“dmesg | grep 04:00.0”,输出“pci 0000:04:00.0: can’t claim BAR 6 [mem 0xfff80000-0xffffffff pref]: no compatible bridge window
    pci 0000:04:00.0: BAR 6: assigned [mem 0xa9080000-0xa90fffff pref]
    pci 0000:04:00.0: Signaling PME through PCIe PME interrupt
    pci-stub 0000:04:00.0: claimed by stub
    vfio-pci 0000:04:00.0: irq 117 for MSI/MSI-X”。不知道是如何 passthough显示不出来,请教一下。

  7. 我的课题也在研究透传问题 ,可以留个联系方式吗 ?

  8. 我的问题是我把dp头插在独显上,已经透传了画面,可是我去设备管理器查看就找不到显卡,装驱动就蓝屏

  9. 大佬 我现在在centos 部署 ,各种问题 ,求解答

发表评论

您的电子邮箱地址不会被公开。