Ongoing virt-who unreliability

Latest response

For a utility that is essentially akin to license DRM, I was expecting that Red Hat would put some effort into making sure virt-who worked seamlessly to limit impact to end users.

I am now working through another list of issues with this tool which is stopping it from updating host to guest mappings in RHN which results in guests being marked as unlicensed. It appears something has broken recently and version 16 is currently returning the following error, without anything else meaningful/useful:

  File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 197, in hypervisorCheckIn
    except rhsm_connection.RateLimitExceededException as e:
AttributeError: 'module' object has no attribute 'RateLimitExceededException'

I attempted to downgrade to version 12, it had a different error, then downgraded to version 10 and it stopped working altogether. In an attempt to resolve the issue (due to bugs found when searching for the errors) I attempted to build an RPM for version 17 from upstream and it appears that the build scripts for version 17 are now broken and no longer build cleanly on RHEL 6.

I am interested to know when I can expect an official build of version 17 for RHEL 6 from Red Hat? in the hope it will resolve the ongoing issues.

Is anyone currently using virt-who successfully? if so, which version, hypervisor and OS platform?

Responses

After some more bugzilla diving, a thread pointed to pysthon-rhsm, after upgrading python-rhsm (Red Hat Subscription Manager), the error has now changed again. These are the combinations of packages and the errors they produce:

-------------------------------------
python-rhsm.x86_64 0:1.14.3-1.el6 
virt-who.noarch 0.16-8.el6
-------------------------------------
Error: AttributeError: 'module' object has no attribute 'RateLimitExceededException'

-------------------------------------
python-rhsm.x86_64 0:1.16.6-1.el6
virt-who.noarch 0.12-10.3.el6_7
-------------------------------------
RestlibException: Organization with id None could not be found.

-------------------------------------
python-rhsm.x86_64 0:1.16.6-1.el6
virt-who.noarch 0.16-8.el6
-------------------------------------
ERROR: Unable to send data: Communication with subscription manager failed with code 4   04: Organization with id None could not be found.

For anyone that has stumbled across this thread, we have found a solution.

Run the following:

subscription-manager identity

This will produce an 'org name' and 'org ID'. If you put the 'ord ID' value into your virt-who configuration under /etc/virt-who.d/config-file in the following format

owner=<number returned>

The organisation ID looks to be fixed. Interestingly, if you use the 'fake' mode (which we were using for testing) it gives the following warning about specifying 'owner' in the configuration file when you execute with --one-shot.

WARNING: Option `owner` is not used in non-hypervisor fake mode

But this isn't correct as the behaviour definitely changes depending on if the owner value is set (ie. it works if the owner matches a valid org ID).

I still believe some of the problems were caused by mismatched versions of 'python-rhsm', 'virt-who' and 'subscription-manager' packages. I think the RPM requirements for the virt-who package need to be revised as currently it doesn't specify high enough minimum versions for the python-rhsm package even though running older versions appears to cause breakage.

From virt-who RPM requires:

python-rhsm >= 1.10.10

PixelDrift.NET Support,

Thanks for detailing your initial problem, diagnostic steps, and most importantly, your solution. With Satellite 6.2 we published the Virtual Instances Guide which details the configuration required for the 'virt-who' agent.

I will follow up the issue about the 'requires' information in the 'virt-who' package and, if needed, raise a Bugzilla ticket to have it updated.

PixelDrift.NET Support,

Further to my previous comment, I'm really curious about the errors you were seeing. If I understand correctly, you solved the issue by using the organisation's numeric ID, not the name, output by the command "subscription-manager identity".

This leads me to ask:

  • Did you previously have the organisation's name specified in the configuration file for the "owner=" parameter? If so, what was the organisation's name, assuming you are willing to share that information?
  • Can you provide the complete configuration file, perhaps before and after, minus any sensitive information?

If you would prefer not to share that information, I'll understand.

Russell,

To be honest we iterated through that many configuration changes I couldn't tell you the state of the config file at a specific point in time, apologies for this.

One issue that I do want to raise is the security/sensitivity aspect of the tool. From our test, it looks like the tool enumerates all VMs in the Virtual Center and then returns this full list back to Red Hat (inlcuding all other OS's). This has raised some eyebrows, especially as this tool should be able to identify Red Hat Linux VMs before shipping the data back.

Can you comment on this behaviour? is this expected? is our understanding of this behaviour correct?

PixelDrift.NET Support,

It's a pity you can't confirm the state of the config file, but I've done similar troubleshooting, so completely understand.

On the topic of security, this has been raised elsewhere in the Discussions area, and it's a fair and reasonable question. I believe, though I would have to confirm, that the virt-who does retrieve the list of all VMs, regardless of operating system. I believe it's only after having retrieved the complete list that it then filters the list to only RHEL instances.

It is possible to restrict the scope of virt-who's data retrieval. One method requires that you grant access only to those hosts which support RHEL instances. If virt-who does not have access to specific VMs or hypervisors, it cannot retrieve their details. The other method involves configuring virt-who to either include or exclude specific hypervisors. For more details see 5.2.1. Limiting the Scope of virt-who Access.

Does this information help? I know it's not a complete answer, but it goes some way to answering your question.

Russell,

This suggestion just isn't practical. On all sites I work on Red Hat VMs are treated like any other VM, there is no segregation of VMware cluster/host based on OS type. In these situations, the VMs also actively migrate between hosts which means limiting VM host visibility is useless.

I don't care that virt-who collects the VM information for other non-Red Hat VMs, I do care if it sends this information back to Red Hat outside the customer network (especially as virt-who can remove this information before it sends it back).

I haven't had a chance to revisit this and look at exactly what is sent back from virt-who.

PixelDrift.NET Support,

In an earlier comment you listed several combinations of virt-who and python-rhsm which resulted in errors. Can you please confirm the package version numbers of those that worked? I would like to include this information in the Bugzilla ticket.

Thank you

Hi Russell,

Currently on the following which is working:

python-rhsm-1.16.6-1.el6.x86_64
virt-who-0.16-8.el6.noarch

This is listed with an error above, but as I mentioned above also, I believe the mismatched versions contributed, but this combination worked when the required options were all provided.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.