Insights shows upgradable packages but dnf disagrees (modules at fault?)

Latest response

Hi folks

Insights is advising me that there are packages available to upgrade on some of my systems:

list of upgradable packages shown in Insights

but:

[root@ipa5 ~]# dnf upgrade
Updating Subscription Management repositories.
Last metadata expiration check: 4:02:27 ago on Tue 12 Oct 2021 06:08:00 UTC.
Dependencies resolved.
Nothing to do.
Complete!

Here's the list of upgradable packages and their versions in insights:

list of upgradable packages in insights

But dnf can only see the installed version of the package, not the "latest version" as displayed by insights:

[root@ipa5 ~]# dnf list --showduplicates perl-HTML-Parser
Updating Subscription Management repositories.
Last metadata expiration check: 0:28:23 ago on Tue 12 Oct 2021 10:12:39 UTC.
Installed Packages
perl-HTML-Parser.x86_64                                                                         3.72-15.module+el8.3.0+6498+9eecfe51                                                                         @rhel-8-for-x86_64-appstream-rpms
Available Packages
perl-HTML-Parser.x86_64                                                                         3.72-15.module+el8.3.0+6498+9eecfe51                                                                         rhel-8-for-x86_64-appstream-rpms 

So far I've managed to figure out that this is something to do with modules. This is terribly non-discoverable (the modules that include a package are absent from the RHEA-2020:4724 advisory, and the output of 'dnf list' and 'dnf info')! But eventually I figured out that the particular perl-libwww-perl package that I have installed is part of the module 'perl-libwww-perl:6.34:8030020200428102957:b967a9a2':

[root@ipa5 ~]# dnf module info perl-libwww-perl:6.34:8030020200428102957:b967a9a2
Updating Subscription Management repositories.
Last metadata expiration check: 0:29:58 ago on Tue 12 Oct 2021 10:12:39 UTC.
Name             : perl-libwww-perl
Stream           : 6.34 [d][e][a]
Version          : 8030020200428102957
Context          : b967a9a2
Architecture     : x86_64
Profiles         : common [d]
Default profiles : common
Repo             : rhel-8-for-x86_64-appstream-rpms
Summary          : A Perl interface to the World-Wide Web
Description      : The libwww-perl collection is a set of Perl modules which provide a simple and consistent application programming interface to the World-Wide Web. The main focus of the library is to provide classes and functions that enable you to write WWW clients. The library also contains modules that are of more general use and even classes that help you implement simple HTTP servers. LWP::Protocol::https adds a support for an HTTPS protocol.
Requires         : perl-IO-Socket-SSL:[2.066]
                 : perl:[5.26]
                 : platform:[el8]
Artifacts        : perl-Data-Dump-0:1.23-7.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-Data-Dump-0:1.23-7.module+el8.3.0+6498+9eecfe51.src
                 : perl-Digest-HMAC-0:1.03-17.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-Digest-HMAC-0:1.03-17.module+el8.3.0+6498+9eecfe51.src
                 : perl-Encode-Locale-0:1.05-10.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-Encode-Locale-0:1.05-10.module+el8.3.0+6498+9eecfe51.src
                 : perl-File-Listing-0:6.04-17.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-File-Listing-0:6.04-17.module+el8.3.0+6498+9eecfe51.src
                 : perl-HTML-Parser-0:3.72-15.module+el8.3.0+6498+9eecfe51.src
                 : perl-HTML-Parser-0:3.72-15.module+el8.3.0+6498+9eecfe51.x86_64
                 : perl-HTML-Parser-debuginfo-0:3.72-15.module+el8.3.0+6498+9eecfe51.x86_64
                 : perl-HTML-Parser-debugsource-0:3.72-15.module+el8.3.0+6498+9eecfe51.x86_64
                 : perl-HTML-Tagset-0:3.20-34.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-HTML-Tagset-0:3.20-34.module+el8.3.0+6498+9eecfe51.src
                 : perl-HTTP-Cookies-0:6.04-2.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-HTTP-Cookies-0:6.04-2.module+el8.3.0+6498+9eecfe51.src
                 : perl-HTTP-Date-0:6.02-19.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-HTTP-Date-0:6.02-19.module+el8.3.0+6498+9eecfe51.src
                 : perl-HTTP-Message-0:6.18-1.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-HTTP-Message-0:6.18-1.module+el8.3.0+6498+9eecfe51.src
                 : perl-HTTP-Negotiate-0:6.01-19.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-HTTP-Negotiate-0:6.01-19.module+el8.3.0+6498+9eecfe51.src
                 : perl-IO-HTML-0:1.001-11.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-IO-HTML-0:1.001-11.module+el8.3.0+6498+9eecfe51.src
                 : perl-LWP-MediaTypes-0:6.02-15.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-LWP-MediaTypes-0:6.02-15.module+el8.3.0+6498+9eecfe51.src
                 : perl-LWP-Protocol-https-0:6.07-4.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-LWP-Protocol-https-0:6.07-4.module+el8.3.0+6498+9eecfe51.src
                 : perl-Mozilla-CA-0:20160104-7.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-Mozilla-CA-0:20160104-7.module+el8.3.0+6498+9eecfe51.src
                 : perl-NTLM-0:1.09-17.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-NTLM-0:1.09-17.module+el8.3.0+6498+9eecfe51.src
                 : perl-Net-HTTP-0
:6.17-2.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-Net-HTTP-0:6.17-2.module+el8.3.0+6498+9eecfe51.src
                 : perl-TimeDate-1:2.30-15.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-TimeDate-1:2.30-15.module+el8.3.0+6498+9eecfe51.src
                 : perl-Try-Tiny-0:0.30-7.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-Try-Tiny-0:0.30-7.module+el8.3.0+6498+9eecfe51.src
                 : perl-WWW-RobotRules-0:6.02-18.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-WWW-RobotRules-0:6.02-18.module+el8.3.0+6498+9eecfe51.src
                 : perl-libwww-perl-0:6.34-1.module+el8.3.0+6498+9eecfe51.noarch
                 : perl-libwww-perl-0:6.34-1.module+el8.3.0+6498+9eecfe51.src

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled, [a]ctive

But there is another module, perl-libwww-perl:6.34:8030020200428102957:7cc0a66d which contains the package that insights lists as the "latest version":

[root@ipa5 ~]# dnf module info perl-libwww-perl:6.34:8030020200428102957:7cc0a66d
Updating Subscription Management repositories.
Last metadata expiration check: 0:34:20 ago on Tue 12 Oct 2021 10:12:39 UTC.
Name             : perl-libwww-perl
Stream           : 6.34 [d][e]
Version          : 8030020200428102957
Context          : 7cc0a66d
Architecture     : x86_64
Profiles         : common [d]
Default profiles : common
Repo             : rhel-8-for-x86_64-appstream-rpms
Summary          : A Perl interface to the World-Wide Web
Description      : The libwww-perl collection is a set of Perl modules which provide a simple and consistent application programming interface to the World-Wide Web. The main focus of the library is to provide classes and functions that enable you to write WWW clients. The library also contains modules that are of more general use and even classes that help you implement simple HTTP servers. LWP::Protocol::https adds a support for an HTTPS protocol.
Requires         : perl-IO-Socket-SSL:[2.066]
                 : perl:[5.24]
                 : platform:[el8]
Artifacts        : perl-Data-Dump-0:1.23-7.module+el8.3.0+6498+31ea7642.noarch
                 : perl-Data-Dump-0:1.23-7.module+el8.3.0+6498+31ea7642.src
                 : perl-Digest-HMAC-0:1.03-17.module+el8.3.0+6498+31ea7642.noarch
                 : perl-Digest-HMAC-0:1.03-17.module+el8.3.0+6498+31ea7642.src
                 : perl-Encode-Locale-0:1.05-10.module+el8.3.0+6498+31ea7642.noarch
                 : perl-Encode-Locale-0:1.05-10.module+el8.3.0+6498+31ea7642.src
                 : perl-File-Listing-0:6.04-17.module+el8.3.0+6498+31ea7642.noarch
                 : perl-File-Listing-0:6.04-17.module+el8.3.0+6498+31ea7642.src
                 : perl-HTML-Parser-0:3.72-15.module+el8.3.0+6498+31ea7642.src
                 : perl-HTML-Parser-0:3.72-15.module+el8.3.0+6498+31ea7642.x86_64
                 : perl-HTML-Parser-debuginfo-0:3.72-15.module+el8.3.0+6498+31ea7642.x86_64
                 : perl-HTML-Parser-debugsource-0:3.72-15.module+el8.3.0+6498+31ea7642.x86_64
                 : perl-HTML-Tagset-0:3.20-34.module+el8.3.0+6498+31ea7642.noarch
                 : perl-HTML-Tagset-0:3.20-34.module+el8.3.0+6498+31ea7642.src
                 : perl-HTTP-Cookies-0:6.04-2.module+el8.3.0+6498+31ea7642.noarch
                 : perl-HTTP-Cookies-0:6.04-2.module+el8.3.0+6498+31ea7642.src
                 : perl-HTTP-Date-0:6.02-19.module+el8.3.0+6498+31ea7642.noarch
                 : perl-HTTP-Date-0:6.02-19.module+el8.3.0+6498+31ea7642.src
                 : perl-HTTP-Message-0:6.18-1.module+el8.3.0+6498+31ea7642.noarch
                 : perl-HTTP-Message-0:6.18-1.module+el8.3.0+6498+31ea7642.src
                 : perl-HTTP-Negotiate-0:6.01-19.module+el8.3.0+6498+31ea7642.noarch
                 : perl-HTTP-Negotiate-0:6.01-19.module+el8.3.0+6498+31ea7642.src
                 : perl-IO-HTML-0:1.001-11.module+el8.3.0+6498+31ea7642.noarch
                 : perl-IO-HTML-0:1.001-11.module+el8.3.0+6498+31ea7642.src
                 : perl-LWP-MediaTypes-0:6.02-15.module+el8.3.0+6498+31ea7642.noarch
                 : perl-LWP-MediaTypes-0:6.02-15.module+el8.3.0+6498+31ea7642.src
                 : perl-LWP-Protocol-https-0:6.07-4.module+el8.3.0+6498+31ea7642.noarch
                 : perl-LWP-Protocol-https-0:6.07-4.module+el8.3.0+6498+31ea7642.src
                 : perl-Mozilla-CA-0:20160104-7.module+el8.3.0+6498+31ea7642.noarch
                 : perl-Mozilla-CA-0:20160104-7.module+el8.3.0+6498+31ea7642.src
                 : perl-NTLM-0:1.09-17.module+el8.3.0+6498+31ea7642.noarch
                 : perl-NTLM-0:1.09-17.module+el8.3.0+6498+31ea7642.src
                 : perl-Net-HTTP-0:6.17-2.module+el8.3.0+6498+31ea7642.noarch
                 : perl-Net-HTTP-0:6.17-2.module+el8.3.0+6498+31ea7642.src
                 : perl-TimeDate-1:2.30-15.module+el8.3.0+6498+31ea7642.noarch
                 : perl-TimeDate-1:2.30-15.module+el8.3.0+6498+31ea7642.src
                 : perl-Try-Tiny-0:0.30-7.module+el8.3.0+6498+31ea7642.noarch
                 : perl-Try-Tiny-0:0.30-7.module+el8.3.0+6498+31ea7642.src
                 : perl-WWW-RobotRules-0:6.02-18.module+el8.3.0+6498+31ea7642.noarch
                 : perl-WWW-RobotRules-0:6.02-18.module+el8.3.0+6498+31ea7642.src
                 : perl-libwww-perl-0:6.34-1.module+el8.3.0+6498+31ea7642.noarch
                 : perl-libwww-perl-0:6.34-1.module+el8.3.0+6498+31ea7642.src

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled, [a]ctive

I've tried to do a bit of reading about modules and contexts, but there isn't a lot to go on. According to https://docs.fedoraproject.org/en-US/modularity/architecture/consuming/dnf-behavior/ the context "is a unique hash value generated by the build system to distinguish binaries built for different Fedora releases as a result of stream expansion". If I'm understanding this correctly, then the version numbers from the two modules above should be considered equal, neither one is newer than the other, and the problem is that Insights doesn't understand that, and ends up displaying the wrong information to me. Is that a known issue?

What I'm still confused by is how insights determines that perl-libwww-perl-0:6.34-1.module+el8.3.0+6498+9eecfe51.noarch is older than perl-libwww-perl-0:6.34-1.module+el8.3.0+6498+31ea7642.noarch; a naive version would determine that perl-libwww-perl-0:6.34-1.module+el8.3.0+6498+9eecfe51.noarch is the newer version, so how does insights decide that it's the older one? I've probably just missed something...

[additional]

Turns out I would have had an easier time identifying which modules provide the package with:

[root@ipa5 ~]# dnf module provides perl-libwww-perl
Updating Subscription Management repositories.
Last metadata expiration check: 1:35:10 ago on Tue 12 Oct 2021 10:12:39 UTC.
perl-libwww-perl-6.34-1.module+el8.3.0+6498+31ea7642.noarch
Module   : perl-libwww-perl:6.34:8030020200428102957:7cc0a66d:x86_64
Profiles : common
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : A Perl interface to the World-Wide Web

perl-libwww-perl-6.34-1.module+el8.3.0+6498+9eecfe51.noarch
Module   : perl-libwww-perl:6.34:8030020200428102957:b967a9a2:x86_64
Profiles : common
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : A Perl interface to the World-Wide Web

perl-libwww-perl-6.34-1.module+el8.3.0+6498+fb59cb73.noarch
Module   : perl-libwww-perl:6.34:8030020200428102957:5839bc99:x86_64
Profiles : common
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : A Perl interface to the World-Wide Web

... although this doesn't show which of the modules are currently enabled/active.

Responses

Hi Sam,

This is a known "issue", please check out the following discussions - I've provided instructions on how to solve the "problem". :)
Basically the solution would be to replace the perl module package versions with the "normal" perl standard package versions.

Product Enhancement Advisory
Older modular RPMs cause erratum
Enchancement discrepency in system

Regards,
Christian

Thanks... I think I'm gonna put up with the spurious upgrade notifications from insights for the time being. This seems like a pretty serious defect in how it interacts with modules which I hope might be fixed some day...

You're welcome, Sam ! I think it will be fixed (some day) ... at least you now know what's going on. :)

Regards,
Christian

Well this may just be a coincidence - but today the perl-related packages have disappeared from insights!

Now insights shows only the 'softhsm' package with updates to apply. That too is a bit of a weird case, but I think the cause is different. I'm told that I can upgrade softhsm from version 2.6.0-5.module+el8.4.0+10227+076cd560.x86_64 to version 2.6.1-5.el8.1.x86_64.

But unlike as with the perl modules, above, there's no mention of this 2.6.1-5.el8.1.x86_64 version of softhsm anywhere on the machine itself:

# dnf list --showduplicates softhsm
Updating Subscription Management repositories.
Last metadata expiration check: 0:02:57 ago on Thu 14 Oct 2021 07:57:46 UTC.
Installed Packages
softhsm.x86_64                                                                             2.6.0-5.module+el8.4.0+10227+076cd560                                                                             @rhel-8-for-x86_64-appstream-rpms
Available Packages
softhsm.x86_64                                                                             2.4.0-2.module+el8.1.0+4098+f286395e                                                                              rhel-8-for-x86_64-appstream-rpms 
softhsm.x86_64                                                                             2.4.0-2.module+el8+2555+b334d87b                                                                                  rhel-8-for-x86_64-appstream-rpms 
softhsm.x86_64                                                                             2.4.0-4.module+el8.2.0+5779+a38c524f                                                                              rhel-8-for-x86_64-appstream-rpms 
softhsm.x86_64                                                                             2.6.0-3.module+el8.3.0+6909+fb33717d                                                                              rhel-8-for-x86_64-appstream-rpms 
softhsm.x86_64                                                                             2.6.0-5.module+el8.4.0+10227+076cd560                                                                             rhel-8-for-x86_64-appstream-rpms 

# dnf module provides softhsm
Updating Subscription Management repositories.
Last metadata expiration check: 0:03:07 ago on Thu 14 Oct 2021 07:57:46 UTC.
softhsm-2.4.0-2.module+el8+2555+b334d87b.x86_64
Module   : idm:DL1:820190227212412:5986f621:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

softhsm-2.4.0-2.module+el8.1.0+4098+f286395e.x86_64
Module   : idm:DL1:8010020190923083643:4a0acb9a:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

softhsm-2.4.0-2.module+el8.1.0+4098+f286395e.x86_64
Module   : idm:DL1:8010020191127093529:6573b795:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

softhsm-2.4.0-4.module+el8.2.0+5779+a38c524f.x86_64
Module   : idm:DL1:8020020200318161616:2ef9514c:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

softhsm-2.6.0-3.module+el8.3.0+6909+fb33717d.x86_64
Module   : idm:DL1:8030020200923172343:9c827e52:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

softhsm-2.6.0-3.module+el8.3.0+6909+fb33717d.x86_64
Module   : idm:DL1:8030020201008125726:fcba9779:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

softhsm-2.6.0-3.module+el8.3.0+6909+fb33717d.x86_64
Module   : idm:DL1:8030020210112201916:fcba9779:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

softhsm-2.6.0-3.module+el8.3.0+6909+fb33717d.x86_64
Module   : idm:DL1:8030020210310163249:fcba9779:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

softhsm-2.6.0-5.module+el8.4.0+10227+076cd560.x86_64
Module   : idm:DL1:8040020210319141752:1f8cbe47:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

softhsm-2.6.0-5.module+el8.4.0+10227+076cd560.x86_64
Module   : idm:DL1:8040020210416132749:5b01ab7e:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

softhsm-2.6.0-5.module+el8.4.0+10227+076cd560.x86_64
Module   : idm:DL1:8040020210526133851:5b01ab7e:x86_64
Profiles : 
Repo     : rhel-8-for-x86_64-appstream-rpms
Summary  : The Red Hat Enterprise Linux Identity Management system module

Some googling drops me at https://centos.pkgs.org/8/epel-x86_64/softhsm-2.6.1-5.el8.1.x86_64.rpm.html which, I suppose, is the source of the package, except...

  • I thought EPEL didn't package things with names that conflict with packages in RHEL?
  • The package isn't appearing in the output of dnf above & I don't know why...

[additional]

Ok I think I see what's going on. I guess enabling a module that contains a package hides non-modular packages with the same name. On another host where I have EPEL installed, but don't have the idm:DL1 module enabled:

[root@xoanon ~]# dnf list softhsm
Updating Subscription Management repositories.
Last metadata expiration check: 1:20:42 ago on Thu 14 Oct 2021 06:50:49 UTC.
Available Packages
softhsm.x86_64                                                                                                       2.6.1-5.el8.1                                                                                                        epel

So at least that what's going on. And wow, that behaviour from DNF sure is confusing and unhelpful. I would have expected 'dnf list --showduplicates' to show the non-modular softhsm so at least I would have had a chance to see that it exists without having to resort to Google!

As for what to do about it--I guess it's another case of Insights not understanding modularity. It shouldn't display an update that would require a modular package to be swapped for a non-modular one.