Why are nsswitch.conf settings being ignored?
Environment
- Red Hat Enterprise Linux
Issue
-
Despite specifying "files" before "dns" in my
nsswitch.conffile and having a valid entry in/etc/hosts, I still see network look-ups. Why? -
The RHEL system often experiences delays in resolving the local hostname listed in the /etc/hosts file, sometimes taking as long as 5, 10, or even 15 seconds to respond.
-
getenttakes longer to resolve the local DNS, even though a local DNS entry for the same name is specified in the/etc/hostsfile.
Resolution
Name look-ups done through the glibc(7) library's gethostbyname(3) function (and similar) obey nsswitch.conf, but this doesn't mean that applications themselves will always adhere to this configuration. This depends entirely on how the application is designed to do name look-ups.
As an example, "getent hosts" will attempt to look up IPv6 addresses before checking to see if an IPv4 entry exists, and if there is no IPv6 entry in /etc/hosts, a network-based look-up will occur.
By design getent will always send query to the DNS server first before checking in the hosts file. If there is an IPV6 DNS entry present in the hosts file for the specific DNS query such behavior is overridden.
Root Cause
If we look at the source code for getent(1), we see that each sub-command has its own set of instructions for how to do look-ups. If we look at the "hosts" command, we'll see this:
if (inet_pton (AF_INET6, key[i], &addr) > 0)
host = gethostbyaddr (addr, IN6ADDRSZ, AF_INET6);
else if (inet_pton (AF_INET, key[i], &addr) > 0)
host = gethostbyaddr (addr, INADDRSZ, AF_INET);
else if ((host = gethostbyname2 (key[i], AF_INET6)) == NULL)
host = gethostbyname2 (key[i], AF_INET);
This indicates that first getent(1) will check to see if the argument passed to it is an IPv6 address. If not, it checks to see if the argument is an IPv4 dotted quad. If it's not, getent(1) next checks for an IPv6 entry for the argument. If it doesn't find one, it finally checks for an IPv4 entry in DNS.
Note that in the latter two steps, gethostbyname2(3) obeys nsswitch.conf settings. With "files dns" specified in nsswitch.conf, a call to gethostbyaddr (addr, IN6ADDRSZ, AF_INET6) will check /etc/hosts first before making a query to external DNS. If there isn't an IPv6 address in /etc/hosts for the name we're looking up, gethostbyname2(3) will do a network-based look-up in order finish its exhaustive search, and that will happen before control passes to gethostbyname2 (key[i], AF_INET).
Diagnostic Steps
- When the
/etc/hostsfile has anipv4local dns entry forexamplehost, dns resolution throughgetenttakes 20 seconds.
[root@node-0 trace]# grep examplehost /etc/hosts
127.0.0.1 examplehost
[root@node-0 trace]# time getent hosts examplehost
127.0.0.1 examplehost
real 0m20.024s
user 0m0.002s
sys 0m0.002s
[root@node-0 trace]# time getent hosts examplehost
127.0.0.1 examplehost
real 0m20.024s
user 0m0.001s
sys 0m0.002s
- As expected, DNS query for
examplehostwas sent to the DNS server192.168.x.xfor 4 consecutive times but all attempts failed due to the dns query getting timed out at the DNS server after waiting for 5 seconds for each attempt.
# grep Timeout ipv6-enable-getent-11.trace.1537 |grep 192.168.x.x
00:11:01.770833 poll([{fd=3<UDP:[10.74.x.x:50903->192.168.x.x:53]>, events=POLLIN}], 1, 5000) = 0 (Timeout) <5.005076>
00:11:06.776147 poll([{fd=3<UDP:[10.74.x.x:50903->192.168.x.x:53]>, events=POLLIN}], 1, 5000) = 0 (Timeout) <5.005064>
00:11:11.781837 poll([{fd=3<UDP:[10.74.x.x:33958->192.168.x.x:53]>, events=POLLIN}], 1, 5000) = 0 (Timeout) <5.005098>
00:11:16.787261 poll([{fd=3<UDP:[10.74.x.x:33958->192.168.x.x:53]>, events=POLLIN}], 1, 5000) = 0 (Timeout) <5.005075>
- After 4 consecutive failures the DNS query was answered from the
/etc/hostsfile.
00:11:21.794451 openat(AT_FDCWD</root/trace>, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 3</etc/hosts> <0.000010>
00:11:21.794498 fstat(3</etc/hosts>, {st_dev=makedev(0xfc, 0x4), st_ino=16911956, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=204, st_atime=1720239061 /* 2024-07-06T00:11:01.769290288-0400 */, st_atime_nsec=769290288, st_mtime=1720239054 /* 2024-07-06T00:10:54.560219448-0400 */, st_mtime_nsec=560219448, st_ctime=1720239054 /* 2024-07-06T00:10:54.562219468-0400 */, st_ctime_nsec=562219468}) = 0 <0.000006>
00:11:21.794579 lseek(3</etc/hosts>, 0, SEEK_SET) = 0 <0.000007> <<<<<<<<<<<
00:11:21.794618 read(3</etc/hosts>, "127.0.0.1 localhost localhost."..., 4096) = 204 <0.000007> <<<<<<<<<<<
00:11:21.794658 read(3</etc/hosts>, "", 4096) = 0 <0.000006>
00:11:21.794692 close(3</etc/hosts>) = 0 <0.000008>
00:11:21.794733 fstat(1</dev/pts/0<char 136:0>>, {st_dev=makedev(0, 0x17), st_ino=3, st_mode=S_IFCHR|0620, st_nlink=1, st_uid=1000, st_gid=5, st_blksize=1024, st_blocks=0, st_rdev=makedev(0x88, 0), st_atime=1720239057 /* 2024-07-06T00:10:57.598472334-0400 */, st_atime_nsec=598472334, st_mtime=1720239057 /* 2024-07-06T00:10:57.598472334-0400 */, st_mtime_nsec=598472334, st_ctime=1720237837 /* 2024-07-05T23:50:37.599472351-0400 */, st_ctime_nsec=599472351}) = 0 <0.000007>
00:11:21.794916 write(1</dev/pts/0<char 136:0>>, "127.0.0.1 examplehost\n", 25) = 25 <0.000136> <<<<<<<<<<<
- After adding an IPV6 DNS entry for
examplehostin the/etc/hostsfile, dns resolution throughgetenttook 2 micro seconds because the query was directly answered from the/etc/hostsfile.
[root@node-0 trace]# vim /etc/hosts
[root@node-0 trace]# grep examplehost /etc/hosts
127.0.0.1 examplehost
2001:db8::1234 examplehost
[root@node-0 trace]# time getent hosts examplehost
2001:db8::1234 examplehost
real 0m0.002s
user 0m0.000s
sys 0m0.002s
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Comments