network cards are not visible/accessible after enabling passthrough

Posted on

Hey,

First of all sorry for the maybe newby question, but I'm kind of new to all of this.
The thing I'm trying to achieve is: running oVirt on our Supermicro Server with GPU's inside and setting up different VM's.
One specific VM should have access to the GPU's directly to be able to do some AI training from Python.
For this reason I'm setting up PCI passthrough, the only problem is that I'm getting stuck quite early during this process.
When I enable the kernel option for the passthrough (intel_iommu=on) and reboot the server, the management interface of oVirt isn't functional anymore.

The hardware that I'm using is:

Supermicro 4029GP-TRT 5U Server
2 x Intel xeon Gold 5217
(1 x) onboard video card based on intel graphics
(2 x) onboard 10G ethernet interfaces
12 x Certified 16GB DDR4 2666MHZ ECC REG
1 x Samsung 480GB PM883
1 x Nvidia RTX 2080Ti OEM card
2 x Nvidia RTX 2080Ti KFA2
1 x Quad Port PCIe Gigabit NIC Startech (ST1000SPEX43)

Then I followed the RHV guide "CHAPTER 1. GPU DEVICE PASSTHROUGH: ASSIGNING A HOST GPU TO A SINGLE VIRTUAL MACHINE"
(URL: https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/html/setting_up_an_nvidia_gpu_for_a_virtual_machine_in_red_hat_virtualization/proc_nvidia_gpu_passthrough_nvidia_gpu_passthrough#Detaching_the_GPU_device_from_the_host_nvidia_gpu_passthrough )

As first step I double checked the BIOS options.
From what I can find online and inside the BIOS is that next features are support by the hardware: Intel VT-d, ACS and SR-IOV.
I also check the BIOS options:
*) Intel VT-d was enabled by default
*) Intel VT directed I/O -> ACS control I enabled manually
*) PCIe/PCI/PnP -> SR-IOV support I enabled manually
*) From what I could read in the BIOS the Interupt remapping was supported and enabled

So I started at the guide section 1 and got stuck. I move the manager VM to another host, enabled the kernel options "intel_iommu=on iommu=pt" in the file "/etc/default/grub", blacklisted nouveau, reboot the system.
And from this moment my management network isn't functional anymore.

I double checked the kernel parameters with this guide of RHV too:
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/html/installation_guide/appe-Configuring_a_Hypervisor_Host_for_PCI_Passthrough#Enabling_IOMMU_Manually

The only thing I didn't try yet is the unsafe interrupt remapping, can this be the problem maybe? (I will try this one as soon as possible anyway)

Does anyone has any clue how this can happen? Is this a problem because I use an PCI-e Startech ethernet card as management interface?
I can imagine when you enable the PCI passthrough that the extra ethernet card will be shared somehow too, sadly I can't find any option to tell the kernel to keep this card for sure for the hypervisor.

The oposite thing I can find in every online guide, how to exclude a PCI card for the hypervisor is done by the "pci-stub" parameter.
As long as I'm stuck at this point I don't believe it's usefull to continu the guide I guess?

Debug commands:

To be sure the config was done well I tried too google options to double check my assumptions above, here you can find some info I got from the oVirt host terminal.

Check Intel VT-d is supported:

"dmesg | grep -e "Directed I/O""

output:
"[ 1.703632] DMAR: Intel(R) Virtualization Technology for Directed I/O"

Check IOMMU and DMAR:

"dmesg | grep -e "IOMMU|DMAR""

output:
[ 0.000000] ACPI: DMAR 000000005fd90578 00290 (v01 ALASKA A M I 00000001 INTL 20091013)
[ 0.000000] DMAR: IOMMU enabled
[ 0.168056] DMAR: Host address width 46
[ 0.168058] DMAR: DRHD base: 0x000000d37fc000 flags: 0x0
[ 0.168065] DMAR: dmar0: reg_base_addr d37fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.168067] DMAR: DRHD base: 0x000000e0ffc000 flags: 0x0
[ 0.168071] DMAR: dmar1: reg_base_addr e0ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.168073] DMAR: DRHD base: 0x000000ee7fc000 flags: 0x0
[ 0.168077] DMAR: dmar2: reg_base_addr ee7fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.168079] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
[ 0.168083] DMAR: dmar3: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.168084] DMAR: DRHD base: 0x000000aaffc000 flags: 0x0
[ 0.168089] DMAR: dmar4: reg_base_addr aaffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.168090] DMAR: DRHD base: 0x000000b87fc000 flags: 0x0
[ 0.168094] DMAR: dmar5: reg_base_addr b87fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.168095] DMAR: DRHD base: 0x000000c5ffc000 flags: 0x0
[ 0.168099] DMAR: dmar6: reg_base_addr c5ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.168101] DMAR: DRHD base: 0x0000009d7fc000 flags: 0x1
[ 0.168105] DMAR: dmar7: reg_base_addr 9d7fc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.168107] DMAR: RMRR base: 0x0000006edcd000 end: 0x0000006edddfff
[ 0.168108] DMAR: ATSR flags: 0x0
[ 0.168109] DMAR: ATSR flags: 0x0
[ 0.168111] DMAR: RHSA base: 0x0000009d7fc000 proximity domain: 0x0
[ 0.168112] DMAR: RHSA base: 0x000000aaffc000 proximity domain: 0x0
[ 0.168113] DMAR: RHSA base: 0x000000b87fc000 proximity domain: 0x0
[ 0.168114] DMAR: RHSA base: 0x000000c5ffc000 proximity domain: 0x0
[ 0.168115] DMAR: RHSA base: 0x000000d37fc000 proximity domain: 0x1
[ 0.168116] DMAR: RHSA base: 0x000000e0ffc000 proximity domain: 0x1
[ 0.168118] DMAR: RHSA base: 0x000000ee7fc000 proximity domain: 0x1
[ 0.168119] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x1
[ 0.168121] DMAR-IR: IOAPIC id 12 under DRHD base 0xc5ffc000 IOMMU 6
[ 0.168122] DMAR-IR: IOAPIC id 11 under DRHD base 0xb87fc000 IOMMU 5
[ 0.168124] DMAR-IR: IOAPIC id 10 under DRHD base 0xaaffc000 IOMMU 4
[ 0.168125] DMAR-IR: IOAPIC id 18 under DRHD base 0xfbffc000 IOMMU 3
[ 0.168126] DMAR-IR: IOAPIC id 17 under DRHD base 0xee7fc000 IOMMU 2
[ 0.168127] DMAR-IR: IOAPIC id 16 under DRHD base 0xe0ffc000 IOMMU 1
[ 0.168129] DMAR-IR: IOAPIC id 15 under DRHD base 0xd37fc000 IOMMU 0
[ 0.168130] DMAR-IR: IOAPIC id 8 under DRHD base 0x9d7fc000 IOMMU 7
[ 0.168132] DMAR-IR: IOAPIC id 9 under DRHD base 0x9d7fc000 IOMMU 7
[ 0.168133] DMAR-IR: HPET id 0 under DRHD base 0x9d7fc000
[ 0.168134] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.170538] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 1.701859] DMAR: dmar6: Using Queued invalidation
[ 1.701866] DMAR: dmar5: Using Queued invalidation
[ 1.701871] DMAR: dmar4: Using Queued invalidation
[ 1.701877] DMAR: dmar3: Using Queued invalidation
[ 1.701883] DMAR: dmar2: Using Queued invalidation
[ 1.701891] DMAR: dmar1: Using Queued invalidation
[ 1.701896] DMAR: dmar0: Using Queued invalidation
[ 1.701901] DMAR: dmar7: Using Queued invalidation
[ 1.703022] DMAR: Hardware identity mapping for device 0000:00:00.0
[ 1.703024] DMAR: Hardware identity mapping for device 0000:00:04.0
[ 1.703025] DMAR: Hardware identity mapping for device 0000:00:04.1
[ 1.703027] DMAR: Hardware identity mapping for device 0000:00:04.2
[ 1.703029] DMAR: Hardware identity mapping for device 0000:00:04.3
[ 1.703030] DMAR: Hardware identity mapping for device 0000:00:04.4
[ 1.703032] DMAR: Hardware identity mapping for device 0000:00:04.5
[ 1.703034] DMAR: Hardware identity mapping for device 0000:00:04.6
[ 1.703035] DMAR: Hardware identity mapping for device 0000:00:04.7
[ 1.703037] DMAR: Hardware identity mapping for device 0000:00:05.0
[ 1.703039] DMAR: Hardware identity mapping for device 0000:00:05.2
[ 1.703040] DMAR: Hardware identity mapping for device 0000:00:05.4
[ 1.703042] DMAR: Hardware identity mapping for device 0000:00:08.0
[ 1.703044] DMAR: Hardware identity mapping for device 0000:00:08.1
[ 1.703045] DMAR: Hardware identity mapping for device 0000:00:08.2
[ 1.703047] DMAR: Hardware identity mapping for device 0000:00:11.0
[ 1.703048] DMAR: Hardware identity mapping for device 0000:00:11.1
[ 1.703050] DMAR: Hardware identity mapping for device 0000:00:11.5
[ 1.703051] DMAR: Hardware identity mapping for device 0000:00:14.0
[ 1.703053] DMAR: Hardware identity mapping for device 0000:00:14.2
[ 1.703055] DMAR: Hardware identity mapping for device 0000:00:16.0
[ 1.703056] DMAR: Hardware identity mapping for device 0000:00:16.1
[ 1.703058] DMAR: Hardware identity mapping for device 0000:00:16.4
[ 1.703059] DMAR: Hardware identity mapping for device 0000:00:17.0
[ 1.703061] DMAR: Hardware identity mapping for device 0000:00:1c.0
[ 1.703063] DMAR: Hardware identity mapping for device 0000:00:1c.5
[ 1.703065] DMAR: Hardware identity mapping for device 0000:00:1f.0
[ 1.703067] DMAR: Hardware identity mapping for device 0000:00:1f.2
[ 1.703068] DMAR: Hardware identity mapping for device 0000:00:1f.4
[ 1.703071] DMAR: Hardware identity mapping for device 0000:00:1f.5
[ 1.703075] DMAR: Hardware identity mapping for device 0000:17:00.0
[ 1.703077] DMAR: Hardware identity mapping for device 0000:17:05.0
[ 1.703079] DMAR: Hardware identity mapping for device 0000:17:05.2
[ 1.703080] DMAR: Hardware identity mapping for device 0000:17:05.4
[ 1.703082] DMAR: Hardware identity mapping for device 0000:17:08.0
[ 1.703084] DMAR: Hardware identity mapping for device 0000:17:08.1
[ 1.703085] DMAR: Hardware identity mapping for device 0000:17:08.2
[ 1.703087] DMAR: Hardware identity mapping for device 0000:17:08.3
[ 1.703089] DMAR: Hardware identity mapping for device 0000:17:08.4
[ 1.703090] DMAR: Hardware identity mapping for device 0000:17:08.5
[ 1.703092] DMAR: Hardware identity mapping for device 0000:17:08.6
[ 1.703094] DMAR: Hardware identity mapping for device 0000:17:08.7
[ 1.703095] DMAR: Hardware identity mapping for device 0000:17:09.0
[ 1.703097] DMAR: Hardware identity mapping for device 0000:17:09.1
[ 1.703099] DMAR: Hardware identity mapping for device 0000:17:09.2
[ 1.703100] DMAR: Hardware identity mapping for device 0000:17:09.3
[ 1.703102] DMAR: Hardware identity mapping for device 0000:17:09.4
[ 1.703104] DMAR: Hardware identity mapping for device 0000:17:09.5
[ 1.703105] DMAR: Hardware identity mapping for device 0000:17:09.6
[ 1.703107] DMAR: Hardware identity mapping for device 0000:17:09.7
[ 1.703109] DMAR: Hardware identity mapping for device 0000:17:0a.0
[ 1.703111] DMAR: Hardware identity mapping for device 0000:17:0a.1
[ 1.703112] DMAR: Hardware identity mapping for device 0000:17:0e.0
[ 1.703114] DMAR: Hardware identity mapping for device 0000:17:0e.1
[ 1.703116] DMAR: Hardware identity mapping for device 0000:17:0e.2
[ 1.703117] DMAR: Hardware identity mapping for device 0000:17:0e.3
[ 1.703119] DMAR: Hardware identity mapping for device 0000:17:0e.4
[ 1.703121] DMAR: Hardware identity mapping for device 0000:17:0e.5
[ 1.703123] DMAR: Hardware identity mapping for device 0000:17:0e.6
[ 1.703124] DMAR: Hardware identity mapping for device 0000:17:0e.7
[ 1.703126] DMAR: Hardware identity mapping for device 0000:17:0f.0
[ 1.703128] DMAR: Hardware identity mapping for device 0000:17:0f.1
[ 1.703130] DMAR: Hardware identity mapping for device 0000:17:0f.2
[ 1.703131] DMAR: Hardware identity mapping for device 0000:17:0f.3
[ 1.703133] DMAR: Hardware identity mapping for device 0000:17:0f.4
[ 1.703135] DMAR: Hardware identity mapping for device 0000:17:0f.5
[ 1.703137] DMAR: Hardware identity mapping for device 0000:17:0f.6
[ 1.703138] DMAR: Hardware identity mapping for device 0000:17:0f.7
[ 1.703140] DMAR: Hardware identity mapping for device 0000:17:10.0
[ 1.703142] DMAR: Hardware identity mapping for device 0000:17:10.1
[ 1.703144] DMAR: Hardware identity mapping for device 0000:17:1d.0
[ 1.703146] DMAR: Hardware identity mapping for device 0000:17:1d.1
[ 1.703147] DMAR: Hardware identity mapping for device 0000:17:1d.2
[ 1.703149] DMAR: Hardware identity mapping for device 0000:17:1d.3
[ 1.703151] DMAR: Hardware identity mapping for device 0000:17:1e.0
[ 1.703153] DMAR: Hardware identity mapping for device 0000:17:1e.1
[ 1.703154] DMAR: Hardware identity mapping for device 0000:17:1e.2
[ 1.703157] DMAR: Hardware identity mapping for device 0000:17:1e.3
[ 1.703159] DMAR: Hardware identity mapping for device 0000:17:1e.4
[ 1.703161] DMAR: Hardware identity mapping for device 0000:17:1e.5
[ 1.703163] DMAR: Hardware identity mapping for device 0000:17:1e.6
[ 1.703165] DMAR: Hardware identity mapping for device 0000:18:00.0
[ 1.703168] DMAR: Hardware identity mapping for device 0000:19:08.0
[ 1.703170] DMAR: Hardware identity mapping for device 0000:19:10.0
[ 1.703173] DMAR: Hardware identity mapping for device 0000:1a:00.0
[ 1.703174] DMAR: Hardware identity mapping for device 0000:1a:00.1
[ 1.703176] DMAR: Hardware identity mapping for device 0000:1a:00.2
[ 1.703178] DMAR: Hardware identity mapping for device 0000:1a:00.3
[ 1.703183] DMAR: Hardware identity mapping for device 0000:1b:00.0
[ 1.703185] DMAR: Hardware identity mapping for device 0000:1b:00.1
[ 1.703187] DMAR: Hardware identity mapping for device 0000:1b:00.2
[ 1.703189] DMAR: Hardware identity mapping for device 0000:1b:00.3
[ 1.703193] DMAR: Hardware identity mapping for device 0000:3a:00.0
[ 1.703195] DMAR: Hardware identity mapping for device 0000:3a:05.0
[ 1.703197] DMAR: Hardware identity mapping for device 0000:3a:05.2
[ 1.703199] DMAR: Hardware identity mapping for device 0000:3a:05.4
[ 1.703202] DMAR: Hardware identity mapping for device 0000:3a:08.0
[ 1.703204] DMAR: Hardware identity mapping for device 0000:3a:09.0
[ 1.703206] DMAR: Hardware identity mapping for device 0000:3a:0a.0
[ 1.703208] DMAR: Hardware identity mapping for device 0000:3a:0a.1
[ 1.703210] DMAR: Hardware identity mapping for device 0000:3a:0a.2
[ 1.703212] DMAR: Hardware identity mapping for device 0000:3a:0a.3
[ 1.703214] DMAR: Hardware identity mapping for device 0000:3a:0a.4
[ 1.703216] DMAR: Hardware identity mapping for device 0000:3a:0a.5
[ 1.703218] DMAR: Hardware identity mapping for device 0000:3a:0a.6
[ 1.703220] DMAR: Hardware identity mapping for device 0000:3a:0a.7
[ 1.703222] DMAR: Hardware identity mapping for device 0000:3a:0b.0
[ 1.703224] DMAR: Hardware identity mapping for device 0000:3a:0b.1
[ 1.703226] DMAR: Hardware identity mapping for device 0000:3a:0b.2
[ 1.703228] DMAR: Hardware identity mapping for device 0000:3a:0b.3
[ 1.703231] DMAR: Hardware identity mapping for device 0000:3a:0c.0
[ 1.703233] DMAR: Hardware identity mapping for device 0000:3a:0c.1
[ 1.703235] DMAR: Hardware identity mapping for device 0000:3a:0c.2
[ 1.703237] DMAR: Hardware identity mapping for device 0000:3a:0c.3
[ 1.703239] DMAR: Hardware identity mapping for device 0000:3a:0c.4
[ 1.703241] DMAR: Hardware identity mapping for device 0000:3a:0c.5
[ 1.703244] DMAR: Hardware identity mapping for device 0000:3a:0c.6
[ 1.703246] DMAR: Hardware identity mapping for device 0000:3a:0c.7
[ 1.703248] DMAR: Hardware identity mapping for device 0000:3a:0d.0
[ 1.703250] DMAR: Hardware identity mapping for device 0000:3a:0d.1
[ 1.703252] DMAR: Hardware identity mapping for device 0000:3a:0d.2
[ 1.703254] DMAR: Hardware identity mapping for device 0000:3a:0d.3
[ 1.703256] DMAR: Hardware identity mapping for device 0000:3b:00.0
[ 1.703259] DMAR: Hardware identity mapping for device 0000:3c:08.0
[ 1.703261] DMAR: Hardware identity mapping for device 0000:3c:10.0
[ 1.703264] DMAR: Hardware identity mapping for device 0000:3d:00.0
[ 1.703266] DMAR: Hardware identity mapping for device 0000:3d:00.1
[ 1.703268] DMAR: Hardware identity mapping for device 0000:3d:00.2
[ 1.703270] DMAR: Hardware identity mapping for device 0000:3d:00.3
[ 1.703274] DMAR: Hardware identity mapping for device 0000:5d:00.0
[ 1.703276] DMAR: Hardware identity mapping for device 0000:5d:02.0
[ 1.703278] DMAR: Hardware identity mapping for device 0000:5d:05.0
[ 1.703281] DMAR: Hardware identity mapping for device 0000:5d:05.2
[ 1.703283] DMAR: Hardware identity mapping for device 0000:5d:05.4
[ 1.703285] DMAR: Hardware identity mapping for device 0000:5d:0e.0
[ 1.703287] DMAR: Hardware identity mapping for device 0000:5d:0e.1
[ 1.703289] DMAR: Hardware identity mapping for device 0000:5d:0f.0
[ 1.703291] DMAR: Hardware identity mapping for device 0000:5d:0f.1
[ 1.703293] DMAR: Hardware identity mapping for device 0000:5d:12.0
[ 1.703295] DMAR: Hardware identity mapping for device 0000:5d:12.1
[ 1.703297] DMAR: Hardware identity mapping for device 0000:5d:12.2
[ 1.703301] DMAR: Hardware identity mapping for device 0000:5d:15.0
[ 1.703303] DMAR: Hardware identity mapping for device 0000:5d:15.1
[ 1.703305] DMAR: Hardware identity mapping for device 0000:5d:16.0
[ 1.703307] DMAR: Hardware identity mapping for device 0000:5d:16.1
[ 1.703309] DMAR: Hardware identity mapping for device 0000:5d:16.4
[ 1.703312] DMAR: Hardware identity mapping for device 0000:5d:16.5
[ 1.703315] DMAR: Hardware identity mapping for device 0000:5e:00.0
[ 1.703317] DMAR: Hardware identity mapping for device 0000:5f:01.0
[ 1.703319] DMAR: Hardware identity mapping for device 0000:5f:02.0
[ 1.703322] DMAR: Hardware identity mapping for device 0000:5f:03.0
[ 1.703324] DMAR: Hardware identity mapping for device 0000:5f:04.0
[ 1.703326] DMAR: Hardware identity mapping for device 0000:60:00.0
[ 1.703329] DMAR: Hardware identity mapping for device 0000:61:00.0
[ 1.703331] DMAR: Hardware identity mapping for device 0000:62:00.0
[ 1.703334] DMAR: Hardware identity mapping for device 0000:63:00.0
[ 1.703337] DMAR: Hardware identity mapping for device 0000:64:00.0
[ 1.703339] DMAR: Hardware identity mapping for device 0000:65:03.0
[ 1.703342] DMAR: Hardware identity mapping for device 0000:66:00.0
[ 1.703344] DMAR: Hardware identity mapping for device 0000:66:00.1
[ 1.703348] DMAR: Hardware identity mapping for device 0000:80:04.0
[ 1.703351] DMAR: Hardware identity mapping for device 0000:80:04.1
[ 1.703353] DMAR: Hardware identity mapping for device 0000:80:04.2
[ 1.703355] DMAR: Hardware identity mapping for device 0000:80:04.3
[ 1.703357] DMAR: Hardware identity mapping for device 0000:80:04.4
[ 1.703359] DMAR: Hardware identity mapping for device 0000:80:04.5
[ 1.703361] DMAR: Hardware identity mapping for device 0000:80:04.6
[ 1.703364] DMAR: Hardware identity mapping for device 0000:80:04.7
[ 1.703367] DMAR: Hardware identity mapping for device 0000:80:05.0
[ 1.703369] DMAR: Hardware identity mapping for device 0000:80:05.2
[ 1.703371] DMAR: Hardware identity mapping for device 0000:80:05.4
[ 1.703373] DMAR: Hardware identity mapping for device 0000:80:08.0
[ 1.703376] DMAR: Hardware identity mapping for device 0000:80:08.1
[ 1.703378] DMAR: Hardware identity mapping for device 0000:80:08.2
[ 1.703382] DMAR: Hardware identity mapping for device 0000:85:00.0
[ 1.703385] DMAR: Hardware identity mapping for device 0000:85:05.0
[ 1.703387] DMAR: Hardware identity mapping for device 0000:85:05.2
[ 1.703389] DMAR: Hardware identity mapping for device 0000:85:05.4
[ 1.703391] DMAR: Hardware identity mapping for device 0000:85:08.0
[ 1.703393] DMAR: Hardware identity mapping for device 0000:85:08.1
[ 1.703396] DMAR: Hardware identity mapping for device 0000:85:08.2
[ 1.703398] DMAR: Hardware identity mapping for device 0000:85:08.3
[ 1.703400] DMAR: Hardware identity mapping for device 0000:85:08.4
[ 1.703403] DMAR: Hardware identity mapping for device 0000:85:08.5
[ 1.703405] DMAR: Hardware identity mapping for device 0000:85:08.6
[ 1.703407] DMAR: Hardware identity mapping for device 0000:85:08.7
[ 1.703409] DMAR: Hardware identity mapping for device 0000:85:09.0
[ 1.703411] DMAR: Hardware identity mapping for device 0000:85:09.1
[ 1.703413] DMAR: Hardware identity mapping for device 0000:85:09.2
[ 1.703415] DMAR: Hardware identity mapping for device 0000:85:09.3
[ 1.703417] DMAR: Hardware identity mapping for device 0000:85:09.4
[ 1.703419] DMAR: Hardware identity mapping for device 0000:85:09.5
[ 1.703422] DMAR: Hardware identity mapping for device 0000:85:09.6
[ 1.703424] DMAR: Hardware identity mapping for device 0000:85:09.7
[ 1.703426] DMAR: Hardware identity mapping for device 0000:85:0a.0
[ 1.703428] DMAR: Hardware identity mapping for device 0000:85:0a.1
[ 1.703430] DMAR: Hardware identity mapping for device 0000:85:0e.0
[ 1.703432] DMAR: Hardware identity mapping for device 0000:85:0e.1
[ 1.703435] DMAR: Hardware identity mapping for device 0000:85:0e.2
[ 1.703437] DMAR: Hardware identity mapping for device 0000:85:0e.3
[ 1.703439] DMAR: Hardware identity mapping for device 0000:85:0e.4
[ 1.703441] DMAR: Hardware identity mapping for device 0000:85:0e.5
[ 1.703443] DMAR: Hardware identity mapping for device 0000:85:0e.6
[ 1.703445] DMAR: Hardware identity mapping for device 0000:85:0e.7
[ 1.703447] DMAR: Hardware identity mapping for device 0000:85:0f.0
[ 1.703450] DMAR: Hardware identity mapping for device 0000:85:0f.1
[ 1.703452] DMAR: Hardware identity mapping for device 0000:85:0f.2
[ 1.703454] DMAR: Hardware identity mapping for device 0000:85:0f.3
[ 1.703456] DMAR: Hardware identity mapping for device 0000:85:0f.4
[ 1.703458] DMAR: Hardware identity mapping for device 0000:85:0f.5
[ 1.703460] DMAR: Hardware identity mapping for device 0000:85:0f.6
[ 1.703462] DMAR: Hardware identity mapping for device 0000:85:0f.7
[ 1.703464] DMAR: Hardware identity mapping for device 0000:85:10.0
[ 1.703466] DMAR: Hardware identity mapping for device 0000:85:10.1
[ 1.703468] DMAR: Hardware identity mapping for device 0000:85:1d.0
[ 1.703471] DMAR: Hardware identity mapping for device 0000:85:1d.1
[ 1.703473] DMAR: Hardware identity mapping for device 0000:85:1d.2
[ 1.703475] DMAR: Hardware identity mapping for device 0000:85:1d.3
[ 1.703477] DMAR: Hardware identity mapping for device 0000:85:1e.0
[ 1.703479] DMAR: Hardware identity mapping for device 0000:85:1e.1
[ 1.703481] DMAR: Hardware identity mapping for device 0000:85:1e.2
[ 1.703483] DMAR: Hardware identity mapping for device 0000:85:1e.3
[ 1.703485] DMAR: Hardware identity mapping for device 0000:85:1e.4
[ 1.703487] DMAR: Hardware identity mapping for device 0000:85:1e.5
[ 1.703490] DMAR: Hardware identity mapping for device 0000:85:1e.6
[ 1.703494] DMAR: Hardware identity mapping for device 0000:ae:00.0
[ 1.703497] DMAR: Hardware identity mapping for device 0000:ae:05.0
[ 1.703499] DMAR: Hardware identity mapping for device 0000:ae:05.2
[ 1.703501] DMAR: Hardware identity mapping for device 0000:ae:05.4
[ 1.703504] DMAR: Hardware identity mapping for device 0000:ae:08.0
[ 1.703506] DMAR: Hardware identity mapping for device 0000:ae:09.0
[ 1.703508] DMAR: Hardware identity mapping for device 0000:ae:0a.0
[ 1.703511] DMAR: Hardware identity mapping for device 0000:ae:0a.1
[ 1.703513] DMAR: Hardware identity mapping for device 0000:ae:0a.2
[ 1.703516] DMAR: Hardware identity mapping for device 0000:ae:0a.3
[ 1.703518] DMAR: Hardware identity mapping for device 0000:ae:0a.4
[ 1.703521] DMAR: Hardware identity mapping for device 0000:ae:0a.5
[ 1.703523] DMAR: Hardware identity mapping for device 0000:ae:0a.6
[ 1.703525] DMAR: Hardware identity mapping for device 0000:ae:0a.7
[ 1.703528] DMAR: Hardware identity mapping for device 0000:ae:0b.0
[ 1.703530] DMAR: Hardware identity mapping for device 0000:ae:0b.1
[ 1.703532] DMAR: Hardware identity mapping for device 0000:ae:0b.2
[ 1.703535] DMAR: Hardware identity mapping for device 0000:ae:0b.3
[ 1.703537] DMAR: Hardware identity mapping for device 0000:ae:0c.0
[ 1.703540] DMAR: Hardware identity mapping for device 0000:ae:0c.1
[ 1.703542] DMAR: Hardware identity mapping for device 0000:ae:0c.2
[ 1.703544] DMAR: Hardware identity mapping for device 0000:ae:0c.3
[ 1.703547] DMAR: Hardware identity mapping for device 0000:ae:0c.4
[ 1.703549] DMAR: Hardware identity mapping for device 0000:ae:0c.5
[ 1.703552] DMAR: Hardware identity mapping for device 0000:ae:0c.6
[ 1.703554] DMAR: Hardware identity mapping for device 0000:ae:0c.7
[ 1.703557] DMAR: Hardware identity mapping for device 0000:ae:0d.0
[ 1.703559] DMAR: Hardware identity mapping for device 0000:ae:0d.1
[ 1.703562] DMAR: Hardware identity mapping for device 0000:ae:0d.2
[ 1.703564] DMAR: Hardware identity mapping for device 0000:ae:0d.3
[ 1.703568] DMAR: Hardware identity mapping for device 0000:d7:00.0
[ 1.703571] DMAR: Hardware identity mapping for device 0000:d7:01.0
[ 1.703574] DMAR: Hardware identity mapping for device 0000:d7:05.0
[ 1.703577] DMAR: Hardware identity mapping for device 0000:d7:05.2
[ 1.703579] DMAR: Hardware identity mapping for device 0000:d7:05.4
[ 1.703582] DMAR: Hardware identity mapping for device 0000:d7:0e.0
[ 1.703584] DMAR: Hardware identity mapping for device 0000:d7:0e.1
[ 1.703587] DMAR: Hardware identity mapping for device 0000:d7:0f.0
[ 1.703589] DMAR: Hardware identity mapping for device 0000:d7:0f.1
[ 1.703592] DMAR: Hardware identity mapping for device 0000:d7:12.0
[ 1.703594] DMAR: Hardware identity mapping for device 0000:d7:12.1
[ 1.703596] DMAR: Hardware identity mapping for device 0000:d7:12.2
[ 1.703598] DMAR: Hardware identity mapping for device 0000:d7:15.0
[ 1.703601] DMAR: Hardware identity mapping for device 0000:d7:15.1
[ 1.703604] DMAR: Hardware identity mapping for device 0000:d7:16.0
[ 1.703607] DMAR: Hardware identity mapping for device 0000:d7:16.1
[ 1.703609] DMAR: Hardware identity mapping for device 0000:d7:16.4
[ 1.703611] DMAR: Hardware identity mapping for device 0000:d7:16.5
[ 1.703612] DMAR: Setting RMRR:
[ 1.703614] DMAR: Ignoring identity map for HW passthrough device 0000:00:14.0 [0x6edcd000 - 0x6edddfff]
[ 1.703618] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 1.703619] DMAR: Ignoring identity map for HW passthrough device 0000:00:1f.0 [0x0 - 0xffffff]
[ 1.703632] DMAR: Intel(R) Virtualization Technology for Directed I/O

Check SVM and VMX:

"grep -E "vmx|svm" /proc/cpuinfo."

output:
see file in attachment

Getting all video cards:

"lspci -nn | grep VGA"

03:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 41)
1a:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti Rev. A] (rev a1)
1b:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)
3d:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti Rev. A] (rev a1)

full dmesg:

see file in attachment

Can anyone help or give usefull tips? (I'm using oVirt V4.3.10)
I'm stuck at this moment, ip a shows the interface still but a ping isn't possible anymore.
My storage VLAN on my 10G bond is still functional and can ping successfull.
Both interface are sadly not showing up with "nmcli conn show" after enabeling the passthrough.

Already many thanks!

Br, Nico

Attachments

Responses