GPU Passthrough on Proxmox for Desktop Streaming

published on 2021-06-13 by C

Introduction

Having recently acquired a secondhand Quadro P400, I wanted to try my hand at having a remote work/gaming pc. Of course the Quadro wouldn't really be suited for gaming, but basic GPU accelerated 3D modeling and the like should be possible. To this end, I wanted to use GPU passthrough in one of my R320s for a Windows VM on Proxmox, withwhatever streaming software seemed the most suitable.

As a baseline, I did try out just using some software through RDP but even the base Windows Desktop experience was kinda choppy for my liking. (VNC can be slightly better, but less well integrated with Windows).

GPU Passthrough/VFIO

The first step was to properly configure the Proxmox host on my R320 to allow the GPU to be passed through. Most of this has been following the excellent guide on (https://pve.proxmox.com/wiki/Pci_passthrough)[https://pve.proxmox.com/wiki/Pci_passthrough].

As a one-liner, this entails isolating the GPU in its own IOMMU group, binding it to the VFIO driver and giving all the correct devices to the VM.

IOMMU

There are plenty of great guides on how to setup IOMMU and the like, so I'll only show the basic options I enabled.

First IOMMU needs to be enabled for boot with intel_iommu=on If you use GRUB to boot (in BIOS mode:) Setting IOMMU enabled in the GRUB_CMDLINE in /etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on"

If you use UFI boot (like I do) you need to set it in /etc/kernel/cmdline:

intel_iommu=on video=efifb:off root=ZFS=rpool/ROOT/pve-1 boot=zfs rootdelay=30

Side note, this system runs on a ZFS boot pool, with a rootdelay set because otherwise it fails to import the zpool intime for the operating system to boot.

After setting the option, the boot options need to be refreshed. On Proxmox this is done with the command pve-efiboot-tool refresh and of course the system needs to be enabled. (For linux/grub I believe you need update-initramfs -u and update-grub)

Once properly enabled you should be able to see in your dmesg output that IOMMU is enabled (dmesg | grep -e DMAR -e IOMMU):

[    0.122930] DMAR: IOMMU enabled

Nice one-liner to print all the IOMMU groups and their devices (not mine, but I don't know where I got it from): for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU Group %s ' "$n"; lspci -nns "${d##*/}"; done;

The relevant entries for my Quadro:

IOMMU Group 19 08:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP107GL [Quadro P400] [10de:1cb3] (rev a1)
IOMMU Group 19 08:00.1 Audio device [0403]: NVIDIA Corporation GP107GL High Definition Audio Controller [10de:0fb9] (rev a1)

Note that it consists of both the actual VGA controller and the audio device.

VFIO

The second step is to bind the VFIO driver. By default the GPU is driven by some sort of 'normal' GPU driver. You can see this in lspci -vv. Example entry for my Quadro (after already binding the vfio driver):

08:00.0 VGA compatible controller: NVIDIA Corporation GP107GL [Quadro P400] (rev a1) (prog-if 00 [VGA controller])
    <snip>
    Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau

Firstly the modules need to be loaded at boot time: /etc/modules/

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

Secondly the VFIO drivers need to be bound to the GPU. This can be a little bit fiddly, sometimes requiring black-listing of the other drivers and other arcane config options. My main source for things to try: https://passthroughpo.st/avada_faq/can-get-vfio-pci-bind-gpu-drivers/.

Now for NICs (like my Mellanox ConnectX-4) I am used to being able to bind drivers without rebooting (which I often used for DPDK in the past). But GPUs seem to be much more temperamental and not so nice. I attempted to bind/unbind via:

echo "0000:08:00.0" > /sys/bus/pci/drivers/vfio-pci/unbind
echo "0000:08:00.0" > /sys/bus/pci/drivers/vfio-pci/bind

But that did not go very well and I recommend just rebooting with other options. Once lspci -vv confirms that the VFIO driver is attached to both the VGA as the audio device you're all good for the next part.

The VM config

Once the IOMMU/VFIO 'stuff' is configured, a VM can be created with access to the PCI device. My vm config looks as follows (having cut out some of the unneeded bits):

args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=proxmox,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_ipi,kvm=off'
balloon: 6048
cores: 6
cpu: host,hidden=1
hostpci0: 08:00,pcie=1,x-vga=1
machine: q35
memory: 8048
numa: 0
ostype: win10
sockets: 1
vga: none

Note that this is my final config, you most likely want to start out with a virtual display connected just so you can do the basic Windows install via Proxmox's built in VNC client. After the basic install, the display can be disconnected and you can use normal VNC/RDP to do some further operations.

Importantly, I did not passthrough any vBIOS images and my VM is BIOS based not UFI, however did this not present any problems as Nvidia has stopped with the ever annoying Code 43 error code.

Windows Streaming Setup

Most of my initial config I did through RDP, however there is a key pitfall that I personally didn't take into account. When using RDP Windows disables/deactivates/messes-with the other GPU (drivers?) in the system (https://www.reddit.com/r/VFIO/comments/hfpm2x/window_10_vm_geforce_experience_and_nvidia_shield/). This means that driver installations and especially shown options can be very hit or miss. I recommend just throwing a VNC server on the system (I used TightVNC). For my Nvidia card I did require to plug in a dummy display, otherwise this is not possible.

I saw 2 potential options for streaming the desktop. The first one was Rainway and the other Moonlight. All my testing was done over Ethernet, with no wireless clients. The client PC was my AMD3700X+1080Ti. This alleviates any potential bottlenecks in both network and client.

Moonlight

Moonlight uses Nvidias Gamestream technology but with an open source client and server system. As it is based on Nvidias Gamestream it requires Geforce/Quadro Experience to be installed, which made me strongly dislike it from the start. I am not a fan of having to log into a piece of driver functionality to make use of my hardware (as online login is required for Quadro Experience). Via RDP Quadro Experience did give me some trouble as it didn't show me all the options I needed, but as noted above, VNC was the solution. Over VNC, the install was very straightforward and after adding mtsc.exe to the games option I could easily stream my desktop. The use of 3D modeling software and some very light gaming worked flawlessly.

Rainway

Rainway is also an open source technology stack, but its functionality is a big more ambiguous. It has an online portal to allow you to stream, for which I could not find a simple local hosted alternative (after minimal looking). A plus however is that the system does not require an Nvidia card and just seem to use the 'normal' video encode functionality found on modern graphics cards. For this, installation and setup was also minimal effort but did require an account as it uses their platform. The streaming experience was just as flawless as with Moonlight.

Conclusion

After having to reboot my server multiple times to set up the GPU passthrough system properly, the setting up of the game stream softwares was remarkably simple and had minimal fuss. Even quickly switching between the two systems and different pieces of running software as about simple and fluent as with native software. Overall I am very impressed with the quality of the provided streams. However, the technology stacks do leave something to be desired. Overall I preferred the experience of Rainway, because I don't need to run Nvidia Gamestream, but I would really like to run it fully locally. As it is open source this might be possible. To more accurately test the streaming performance I do need to check the performance on my laptop over a wireless connection as this may proof to give the edge to Moonlight.

The next step will likely be acquiring a more powerful GPU and getting a fatter server to allow me to set up a more complete remote workstation. However with the current GPU prices this may have to wait a while. Another next possibility is trying to passthrough the GPU to my LXC containers for some CUDA/AI experimentation

References