use Host GPU in Virtual Machines

In this Scenario we use following Componets:

Mainboard with AMD CPU AM4 with no iGPU

Proxmox VE with uefi-grub

only one GPU (rtx4060)

1.Bios Settings

enable "AMD-V" (Virtualisation)
enable "IOMMU" (important for Isolation and Passthrough PCI Devices)
enable "ACL" (Access Control for IOMMU) if aviable
disable "CMS" (Compatibility Mode use only if GPU supports UEFI and you also have to use UEFI Boot)

optionaly enable "Resizeble Bar Above 4G" (for maximum Performance for GPU)

2. On Proxmox host edit the Grub Config and add additionaly Parameters for IOMMU and also to disable framebuffer so the Host not using the GPU and not blocking it for the VM.

nano /etc/default/grub

#edit following line and add following after "quiet" (intel Cpu have different iommu options!")

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt initcall_blacklist=sysfb_init nofb nomodeset video=vesafb:off,efifb:off"

# after that update grub

update-grub

3. Then Block following Kernel Modules for Nvidia

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf

#and add folowing moduls in /etc/modprobe.d/blacklsit.conf
                                   
blacklist nouveau
blacklist nvidia*
options nouveau modeset=0

run this command and write down the id for GPU and AUDIO (same IOMMU Group) and block it also

lspci -v | grep NVIDIA

09:00.0 VGA compatible controller: NVIDIA Corporation AD107 [GeForce RTX 4060] (rev a1) (prog-if 00 [VGA controller])
09:00.1 Audio device: NVIDIA Corporation AD107 High Definition Audio Controller (rev a1)

# write the ID you copied above step
lspci -n -s 09:00

09:00.0 0300: 10de:2882 (rev a1)
09:00.1 0403: 10de:22be (rev a1)

#and also following to "/etc/modprobe.d/vfio.conf"                                      
options vfio-pci ids=10de:2882,10de:22be disable_vga=1

Add folowing Kernelmoduls to enable the driver for GPU PAssthrough

#append below lines to "/etc/modules"
vifio
vifio_iommu_type1
vifio_pci
vifio_virqfd

run following command and reboot

update-initramfs -u -k all
reboot

Congratulations your Proxmox host should now be ready to create VM and add RAW PCI DEVICE PASSTHROUGH

Make sure to use CPU q35 and host best look a Youtube Video on how to create a Windows or Linux VM for adding PCI-GPU and use as Primary Display Driver.

Also you need SUNSHINE/MOONLIGHT or PARSEC to Access The Vm (build in VNC in Proxmox will no longer work)