rpcbind fails to start with IPv6 disabled

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux 7.3 new install
  • net.ipv6.conf.all.disable_ipv6 = 1 in /etc/sysctl.conf

Issue

  • On RHEL7.3 GA systems with ipv6 disabled, rpcbind fails to start.
  • #systemctl status rpcbind.socket shows error Failed to listen on RPCbind Server Activation Socket.

Resolution

  • Rebuild the initramfs: # dracut -v -f to incorporate the changes made to /etc/sysctl.conf
  • It is necessary to rebuild the initramfs when ever there are modifications made to /etc/sysctl.conf.
  • Following the rebuild of the initramfs please reboot the system in order for the systemd response to take effect.
  • For additional reading please read When is it necessary to rebuild the initramfs?

Root Cause

  • This problem is believed to be caused by omitting the step to update the initramfs per How do I disable or enable the IPv6 protocol in Red Hat Enterprise Linux?
  • Without systemd being aware that ipv6 is disabled in the early boot process, it will attempt to make a connection on port 111 on any ipv6 address, [::]:111, due to updates made to rpcbind per BZ 1359592.
# rpm -q rpcbind --changelog | head
* Tue Aug 02 2016 Steve Dickson <steved@redhat.com> - 0.2.0-38
- Removing the braces from the ${RPCBIND_ARGS} in rpcbind.service (bz 1362232)

* Fri Jul 29 2016 Steve Dickson <steved@redhat.com> - 0.2.0-37
- Make sure rpcbind.socket listens for remote IPv6 connections (bz 1359592)

* Thu Jul 21 2016 Steve Dickson <steved@redhat.com> - 0.2.0-36
- Added upstream debugging (bz 1358890)

* Sat Apr 09 2016 Steve Dickson <steved@redhat.com> - 0.2.0-35
  • rpcbind.socket service file for rpcbind-0.2.0-38.el7.x86_64 released with Red Hat Enterprise Linux Server release 7.3
# systemctl cat rpcbind.socket
# /usr/lib/systemd/system/rpcbind.socket
[Unit]
Description=RPCbind Server Activation Socket

[Socket]
ListenStream=/var/run/rpcbind.sock
ListenStream=[::]:111
ListenStream=0.0.0.0:111
BindIPv6Only=ipv6-only

[Install]
WantedBy=sockets.target
  • rpcbind.socket service file for rpcbind-0.2.0-32.el7.x86_64 released with Red Hat Enterprise Linux Server release 7.2
# systemctl cat rpcbind.socket
# /usr/lib/systemd/system/rpcbind.socket
[Unit]
Description=RPCbind Server Activation Socket

[Socket]
ListenStream=/var/run/rpcbind.sock

[Install]
WantedBy=sockets.target

Diagnostic Steps

  • rpcbind.socket status before disabling ipv6
# systemctl status rpcbind.socket
● rpcbind.socket - RPCbind Server Activation Socket
   Loaded: loaded (/usr/lib/systemd/system/rpcbind.socket; enabled; vendor preset: enabled)
   Active: active (listening) since Wed 2016-12-07 13:47:22 EST; 1min 49s ago
   Listen: /var/run/rpcbind.sock (Stream)
           [::]:111 (Stream)
           0.0.0.0:111 (Stream)

Dec 07 13:47:22 rhel73-nfs-client systemd[1]: Listening on RPCbind Server Activation Socket.
Dec 07 13:47:22 rhel73-nfs-client systemd[1]: Starting RPCbind Server Activation Socket
  • After adding net.ipv6.conf.all.disable_ipv6 = 1 to /etc/sysctl.conf and issuing # systemctl reboot
# systemctl status rpcbind.socket
● rpcbind.socket - RPCbind Server Activation Socket
   Loaded: loaded (/usr/lib/systemd/system/rpcbind.socket; enabled; vendor preset: enabled)
   Active: failed (Result: resources)
   Listen: /var/run/rpcbind.sock (Stream)
           [::]:111 (Stream)
           0.0.0.0:111 (Stream)

Dec 07 13:50:56 rhel73-nfs-client systemd[1]: rpcbind.socket failed to listen on sockets: Address family not supported by protocol
Dec 07 13:50:56 rhel73-nfs-client systemd[1]: Failed to listen on RPCbind Server Activation Socket.
Dec 07 13:50:56 rhel73-nfs-client systemd[1]: Unit rpcbind.socket entered failed state.
Dec 07 13:50:56 rhel73-nfs-client systemd[1]: Starting RPCbind Server Activation Socket.
Dec 07 13:50:56 rhel73-nfs-client systemd[1]: rpcbind.socket failed to listen on sockets: Address family not supported by protocol
Dec 07 13:50:56 rhel73-nfs-client systemd[1]: Failed to listen on RPCbind Server Activation Socket.
Dec 07 13:50:56 rhel73-nfs-client systemd[1]: Starting RPCbind Server Activation Socket.
  • After rebuilding the initramfs with # dracut -v -f and issuing # systemctl reboot
● rpcbind.socket - RPCbind Server Activation Socket
   Loaded: loaded (/usr/lib/systemd/system/rpcbind.socket; enabled; vendor preset: enabled)
   Active: active (listening) since Wed 2016-12-07 16:04:16 EST; 12s ago
   Listen: /var/run/rpcbind.sock (Stream)
           0.0.0.0:111 (Stream)

Dec 07 16:04:16 rhel73-nfs-client systemd[1]: Listening on RPCbind Server Activation Socket.
Dec 07 16:04:16 rhel73-nfs-client systemd[1]: Starting RPCbind Server Activation Socket.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

5 Comments

Hi,

Shouldn't -H also be passed to dracut so /etc/sysctl.conf is copied into the initial RAM filesystem?

# uname -r
3.10.0-514.2.2.el7.x86_64
# dracut -v -f > /dev/null 2>&1
# lsinitrd /boot/initramfs-3.10.0-514.2.2.el7.x86_64.img -f /etc/sysctl.conf
# dracut -v -f -H > /dev/null 2>&1
# lsinitrd /boot/initramfs-3.10.0-514.2.2.el7.x86_64.img -f /etc/sysctl.conf
# System default settings live in /usr/lib/sysctl.d/00-system.conf.
# To override those settings, enter new settings here, or in an /etc/sysctl.d/<name>.conf file
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv6.conf.all.disable_ipv6 = 1

It is not required to use the -H flag to incorporate the changes into the initramfs.

Will there be a fix for this behavior? Having to update initramfs for a setting change in sysctl.conf doesn't make much sense. Why does rpcbind assume IPV6 is enabled?

Hello Matt,

"Will there be a fix for this behavior?" o The behavior is expected behavior where IPv6 is not properly disabled either via a grub boot parameter or disabling the protocol via a sysctl.

"Having to update initramfs for a setting change in sysctl.conf doesn't make much sense. " o /etc/sysctl.conf && /etc/sysctl.d are read during the early boot process. If IPv6 is disabled via a sysctl the system has to be informed of such disabling during the early boot process particularly due to SystemD starting much earlier that previous init systems.

"Why does rpcbind assume IPV6 is enabled?" o As stated above, the change in behavior is due to an update in rpcbind * Fri Jul 29 2016 Steve Dickson steved@redhat.com - 0.2.0-37 - Make sure rpcbind.socket listens for remote IPv6 connections (bz 1359592)

6/14/2018 - We currently are running RHEL 7.4. The rebuilding of initramfs doesn't seem to solve the issue. Disabling via grub does seem to do the trick though.