virt-who reports a different UUID of ESX VM than what’s shown in dmidecode for VMs

Solution Verified - Updated -

Environment

  • Red Hat Satellite 6.x
  • virt-who
  • VCenter 6.5
  • Subscription watch

Issue

  • Why is the UUID reported by virt-who different from dmidecode of VM?
  • The UUID reported by virt-who differs from the dmidecode of VMs on VMware, causing overutilization issues in Subscription Watch.
  • Why the VMs are reported as standalone in subscription watch inventory even though the virt-who is set up properly?

Resolution

  • There is a known problem where virt-who is reporting a different UUID than what’s shown in dmidecode for VMs running on VMware. Due to the UUID mismatch, the host-guests mapping are not available for the VM and VMs are reported as standalone in subscription watch. The issue is tracked via JIRA #RHEL-45636.

  • From VMWare side, when upgrading your VM hardware version, you should also be adding and/or changing the configuration entry below to TRUE

    From

    acpi.smbiosVersion2.7 = "FALSE"

    To

    acpi.smbiosVersion2.7 = "TRUE"

  • For more KB articles/solutions related to Virt-who and Virtual Datacenter (VDC) Subscriptions Issues, please refer to the Consolidated Troubleshooting Article for Virt-who and Virtual Datacenter (VDC) Subscriptions Issues

Root Cause

  • The UUID reported by VMware HW > =13 is not as per RFC 4122 standards. Red Hat reports UUID per RFC 4122 standards. Find below examples:
[root@SMBIOS 2.6 client ~]# dmidecode |head -n 3
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.

# dmidecode|grep -A 5 '^System Information'
System Information
    Manufacturer: VMware, Inc.
    Product Name: VMware Virtual Platform
    Version: None
    Serial Number: VMware-42 19 a0 7a fd ee d5 09-5f b8 4b 7e 5c 2d 92 c5     <<=====
    UUID: 7aa01942-eefd-09d5-5fb8-4b7e5c2d92c5                                              <<=====

The 'dmidecode' command tries to swap the bytes in the first 3 fields if the system's SMBIOS version is 2.6 or greater.

Eg:

SMBIOS 2.7 present

Serial Number: VMware-42 19 a0 7a fd ee d5 09-5f b8 4b 7e 5c 2d 92 c5

SMBIOS >= 2.6

  7a    a0    19    42    ee    fd    09    d5    5f    b8     4b     7e     5c     2d     92     c5
p[3], p[2], p[1], p[0], p[5], p[4], p[7], p[6], p[8], p[9], p[10], p[11], p[12], p[13], p[14], p[15]

UUID: 7aa01942-eefd-09d5-5fb8-4b7e5c2d92c5

[root@SMBIOS 2.4 client ~]# dmidecode |head -n 3
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 2.4 present.

# dmidecode|grep -A 5 '^System Information'
System Information
    Manufacturer: VMware, Inc.
    Product Name: VMware Virtual Platform
    Version: None
    Serial Number: VMware-42 25 0e e6 f5 eb 85 76-46 f1 f1 f5 c1 9c bf 2a     <<=====
    UUID: 42250ee6-f5eb-8576-46f1-f1f5c19cbf2a                                              <<=====


The 'dmidecode' command doesn't swap the bytes in the first 3 fields if the system's SMBIOS version is less than 2.6.

Eg:

SMBIOS 2.4 present.

Serial Number: VMware-42 25 0e e6 f5 eb 85 76-46 f1 f1 f5 c1 9c bf 2a

SMBIOS < 2.6

  42    25    0e    e6    f5    eb    85    76    46    f1     f1     f5     c1     9c     bf     2a
p[0], p[1], p[2], p[3], p[4], p[5], p[6], p[7], p[8], p[9], p[10], p[11], p[12], p[13], p[14], p[15]

UUID: 42250ee6-f5eb-8576-46f1-f1f5c19cbf2a

There is also another approach that could cause a similar behavior, but before, let me present some standard values when creating virtual machines on VMWare.

For all the new servers created on VMWare, they will be like the below when installing RHEL

VM Hardware Version SMBIOS Version
< 12 SMBIOS 2.4
> = 12 SMBIOS 2.7

However, if the VM is created in version 13 for example, and the sysadmin proceeds with the hardware upgrade via webUI/vCenter to keep the compatibility with the hypervisor, a new flag will be added automatically in the VMX file. You can see the entry below

acpi.smbiosVersion2.7 = "FALSE"

After this change, VMs with hardware version > = 12 will be reported as old hardware versions, presenting the SMBIOS 2.4 instead of SMBIOS 2.7 and as consequence causing issues related to the virt-who host-guest mapping.

Diagnostic Steps

  • While troubleshooting for virt-who's host-guest mapping search using VCenter's UUID:
"facts": {
                "hypervisor.type": "VMware ESXi", 
                "dmi.system.uuid": "4c4c4544-004c-3510-8034-b9c04f514d32", 
                "cpu.cpu_socket(s)": "2", 
                "hypervisor.cluster": "Cluster4", 
                "hypervisor.version": "6.5.0"
            }
        }, 
        {
            "hypervisorId": {
                "hypervisorId": "esx1.redhat.com"
            }, 
            "name": "esx1.redhat.com", 
            "guestIds": [
{
                    "guestId": "421250eb-e096-9c97-8e46-121d5977a347",      <<====== UUID of VM as reported by VCenter
                    "state": 1, 
                    "attributes": {
                        "active": 1, 
                        "virtWhoType": "esx"
                    }
                },

Also, when creating a new VM in version 14 for example, you can see via dmidecode command that the BIOS will be 2.7. However, after upgrading the hardware compatibility of the VM, when reruning the dmidecode you will see the BIOS version 2.4, which means, the acpi.smbiosVersion2.7 it's TRUE by default when creating new VMs.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments