chronyc sources not working for normal users

Latest response

Hi All,

Is anyone across the below issue in which the chronyc sources (or other chronyc cmds) are not working from non-privileged users. Although it perfectly works fine from root.

$ chronyc sources
506 Cannot talk to daemon
$ chronyc -d sources
Resolved 127.0.0.1 to 127.0.0.1
Resolved ::1 to ::1
Opening connection to /var/run/chrony/chronyd.sock
Could not bind socket : Permission denied
Opening connection to 127.0.0.1:323
Sent 32 bytes
Timeout 0.999928 seconds
Sent 32 bytes
Timeout 1.999979 seconds
Sent 32 bytes
Timeout 3.999973 seconds
Opening connection to ::1:323
Could not create socket : Address family not supported by protocol
506 Cannot talk to daemon
$

I tried googling around but still cant find the solution.

Regards,
Sandeep

Responses

Hi Sundeep,

Based on your report, I think the chronyd might not be running. That is the cause of your problems, not the permissions on the socket file.

Error code 506 points to daemon not running. Check it with:

$ chronyc tracking
506 Cannot talk to daemon

Here is the proof from one of my servers:

$ ls -als /var/run/chrony/chronyd.sock
ls: cannot access /var/run/chrony/chronyd.sock: Permission denied

$ chronyc sources
210 Number of sources = 6
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
(list of servers undisclosed to public)

$ pgrep chronyd
1268

$ chronyc tracking
Reference ID    : C2B8720D (myhost.domain.dom)
Stratum         : 2
Ref time (UTC)  : Fri Sep 06 01:24:42 2019
System time     : 0.000000021 seconds slow of NTP time
Last offset     : +0.000105018 seconds
RMS offset      : 0.000105018 seconds
Frequency       : 20.913 ppm slow
Residual freq   : -2.571 ppm
Skew            : 0.011 ppm
Root delay      : 0.000991902 seconds
Root dispersion : 0.001081072 seconds
Update interval : 0.0 seconds
Leap status     : Normal

Please verify if chronyd service is running:

$ pgrep chronyd

$ systemctl status chronyd -l

$ chronyc tracking

Regards,

Dusan Baljevic (amateur radio VK2COT)

Hi Dusan,

The service is running. Please see below From non-privileged user it doesn't work $ chronyc sources 506 Cannot talk to daemon $ logout

From root it works:

chronyc sources 210 Number of sources = 4

MS Name/IP address Stratum Poll Reach LastRx Last sample ^+ 10 1 10 377 801 -126us[ -140us] +/- 1515us ^* 10 1 10 377 523 -133us[ -148us] +/- 1609us ^+ 10 1 10 377 229 +655ns[ +655ns] +/- 1580us ^+ 10 1 10 377 54 -191us[ -191us] +/- 1679us

ps -ef | grep -i chrony root 334536 334051 0 01:31 pts/1 00:00:00 grep --color=auto -i chrony chrony 493271 1 0 Sep05 ? 00:00:00 /usr/sbin/chronyd -4

systemctl status chronyd ● chronyd.service - NTP client/server Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-09-05 05:05:46 UTC; 20h ago Docs: man:chronyd(8) man:chrony.conf(5) Process: 493272 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS) Process: 493269 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS) Main PID: 493271 (chronyd) CGroup: /system.slice/chronyd.service └─493271 /usr/sbin/chronyd -4

Sep 05 05:05:46 gbp4520s systemd[1]: Starting NTP client/server... Sep 05 05:05:46 gbp4520s chronyd[493271]: chronyd version 3.2 starting (+CMDMON +NTP +REFCLOCK...UG) Sep 05 05:05:46 gbp4520s chronyd[493271]: commandkey directive is no longer supported Sep 05 05:05:46 gbp4520s chronyd[493271]: generatecommandkey directive is no longer supported Sep 05 05:05:46 gbp4520s chronyd[493271]: Frequency -12.858 +/- 0.019 ppm read from /var/lib/c...ift Sep 05 05:05:46 gbp4520s systemd[1]: Started NTP client/server. Sep 05 05:05:51 gbp4520s chronyd[493271]: Selected source 10 Sep 05 05:06:57 gbp4520s chronyd[493271]: Selected source 10 Hint: Some lines were ellipsized, use -l to show in full. #

systemctl status chronyd -l ● chronyd.service - NTP client/server Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-09-05 05:05:46 UTC; 20h ago Docs: man:chronyd(8) man:chrony.conf(5) Process: 493272 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS) Process: 493269 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS) Main PID: 493271 (chronyd) CGroup: /system.slice/chronyd.service └─493271 /usr/sbin/chronyd -4

Sep 05 05:05:46 gbp4520s systemd[1]: Starting NTP client/server... Sep 05 05:05:46 gbp4520s chronyd[493271]: chronyd version 3.2 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SECHASH +SIGND +ASYNCDNS +IPV6 +DEBUG) Sep 05 05:05:46 gbp4520s chronyd[493271]: commandkey directive is no longer supported Sep 05 05:05:46 gbp4520s chronyd[493271]: generatecommandkey directive is no longer supported Sep 05 05:05:46 gbp4520s chronyd[493271]: Frequency -12.858 +/- 0.019 ppm read from /var/lib/chrony/drift Sep 05 05:05:46 gbp4520s systemd[1]: Started NTP client/server. Sep 05 05:05:51 gbp4520s chronyd[493271]: Selected source 10 Sep 05 05:06:57 gbp4520s chronyd[493271]: Selected source 10

Hi Dusan,

The service is running. Please see below From non-privileged user it doesn't work $ chronyc sources 506 Cannot talk to daemon $ logout

From root it works:

chronyc sources

210 Number of sources = 4

MS Name/IP address Stratum Poll Reach LastRx Last sample

^+ 10 1 10 377 801 -126us[ -140us] +/- 1515us ^* 10 1 10 377 523 -133us[ -148us] +/- 1609us ^+ 10 1 10 377 229 +655ns[ +655ns] +/- 1580us ^+ 10 1 10 377 54 -191us[ -191us] +/- 1679us

ps -ef | grep -i chrony

root 334536 334051 0 01:31 pts/1 00:00:00 grep --color=auto -i chrony chrony 493271 1 0 Sep05 ? 00:00:00 /usr/sbin/chronyd -4

systemctl status chronyd

● chronyd.service - NTP client/server Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-09-05 05:05:46 UTC; 20h ago Docs: man:chronyd(8) man:chrony.conf(5) Process: 493272 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS) Process: 493269 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS) Main PID: 493271 (chronyd) CGroup: /system.slice/chronyd.service └─493271 /usr/sbin/chronyd -4

Sep 05 05:05:46 gbp4520s systemd[1]: Starting NTP client/server... Sep 05 05:05:46 gbp4520s chronyd[493271]: chronyd version 3.2 starting (+CMDMON +NTP +REFCLOCK...UG) Sep 05 05:05:46 gbp4520s chronyd[493271]: commandkey directive is no longer supported Sep 05 05:05:46 gbp4520s chronyd[493271]: generatecommandkey directive is no longer supported Sep 05 05:05:46 gbp4520s chronyd[493271]: Frequency -12.858 +/- 0.019 ppm read from /var/lib/c...ift Sep 05 05:05:46 gbp4520s systemd[1]: Started NTP client/server. Sep 05 05:05:51 gbp4520s chronyd[493271]: Selected source 10 Sep 05 05:06:57 gbp4520s chronyd[493271]: Selected source 10 Hint: Some lines were ellipsized, use -l to show in full. #

systemctl status chronyd -l

● chronyd.service - NTP client/server Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-09-05 05:05:46 UTC; 20h ago Docs: man:chronyd(8) man:chrony.conf(5) Process: 493272 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS) Process: 493269 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS) Main PID: 493271 (chronyd) CGroup: /system.slice/chronyd.service └─493271 /usr/sbin/chronyd -4

Sep 05 05:05:46 gbp4520s systemd[1]: Starting NTP client/server... Sep 05 05:05:46 gbp4520s chronyd[493271]: chronyd version 3.2 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SECHASH +SIGND +ASYNCDNS +IPV6 +DEBUG) Sep 05 05:05:46 gbp4520s chronyd[493271]: commandkey directive is no longer supported Sep 05 05:05:46 gbp4520s chronyd[493271]: generatecommandkey directive is no longer supported Sep 05 05:05:46 gbp4520s chronyd[493271]: Frequency -12.858 +/- 0.019 ppm read from /var/lib/chrony/drift Sep 05 05:05:46 gbp4520s systemd[1]: Started NTP client/server. Sep 05 05:05:51 gbp4520s chronyd[493271]: Selected source 10 Sep 05 05:06:57 gbp4520s chronyd[493271]: Selected source 10

Hi again,

Interesting.

From the on-line manuals:

There are two ways chronyc can access chronyd. One is the Internet Protocol (IPv4 or IPv6) and the other is a Unix domain socket, which is accessible locally by the root or chrony user. By default, chronyc first tries to connect to the Unix domain socket. The compiled-in default path is /var/run/chrony/chronyd.sock. If that fails (e.g. because chronyc is running under a non-root user), it will try to connect to 127.0.0.1 and then ::1.

Only the following monitoring commands, which do not affect the behaviour of chronyd, are allowed from the network: activity, manual list, rtcdata, smoothing, sources, sourcestats, tracking, waitsync. The set of hosts from which chronyd will accept these commands can be configured with the cmdallow directive in the chronyd's configuration file or the cmdallow command in chronyc. By default, the commands are accepted only from localhost (127.0.0.1 or ::1).

All other commands are allowed only through the Unix domain socket. When sent over the network, chronyd will respond with a "Not authorised" error, even if it is from localhost. In chrony versions before 2.2 they were allowed from the network if they were authenticated with a password, but that is no longer supported.

Can you check loopback interface on your server:

$ awk '/^127.0.0.1|::1/ {print}' /etc/hosts

Also, check if loopback interface is up. You get error 506 for chronyc command if the interface is down:

$ ip a

This would be a bad result:

1: lo: <LOOPBACK> mtu 65536 qdisc noqueue state DOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

... and this would be OK:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo

Regards,

Dusan Baljevic (amateur radio VK2COT)

Hi Dusan,

The loopback address seems fine.

# awk '/^127.0.0.1|::1/ {print}' /etc/hosts 127.0.0.1 localhost 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 [root@gbp4520s ~] #

Hi again :)

We did not get confirmation if loopback infterface is up and running?

Secondly, can you try this for debugging:

$ strace chronyc tracking

I expect to see something like this at the end of trace:

getsockname(3, {sa_family=AF_INET, sin_port=htons(18242), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
close(3)                                = 0
exit_group(0)                           = ?
+++ exited with 0 +++

When it does not work, you might get:

getsockname(3, {sa_family=AF_INET, sin_port=htons(36521), sin_addr=inet_addr("MY.IP.ADD.RESS")}, [16]) = 0
close(3)                                = 0
socket(AF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET6, sin6_port=htons(323), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 ENETUNREACH (Network is unreachable)
getsockname(3, {sa_family=AF_INET6, sin6_port=htons(26606), inet_pton(AF_INET6, "::", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
close(3)                                = 0
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fe0c7fd0000
write(1, "506 Cannot talk to daemon\n", 26506 Cannot talk to daemon
) = 26
exit_group(1)                           = ?
+++ exited with 1 +++

Regards,

Dusan Baljevic (amateur radio VK2COT)

Hi Dusan,

Apologies for delayed response, but we have migrated this server to ptp.

But, interestingly got the similar error for root user as well in another server. I ran strace on that server and getting the below timeouts for loopback ip.

munmap(0x7f3fc15df000, 40033) = 0 open("/proc/sys/crypto/fips_enabled", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3fc15e8000 read(3, "0\n", 1024) = 2 close(3) = 0 munmap(0x7f3fc15e8000, 4096) = 0 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3 setsockopt(3, SOL_IP, IP_RECVERR, [1], 4) = 0 sendto(3, "\6\1\0\0\0\16\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 48, 0, {sa_family=AF_INET, sin_port=htons(323), sin_addr=inet_addr("127.0.0.1")}, 16) = 48 select(4, [3], [], [], {0, 100000}) = 0 (Timeout) sendto(3, "\6\1\0\0\0\16\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 48, 0, {sa_family=AF_INET, sin_port=htons(323), sin_addr=inet_addr("127.0.0.1")}, 16) = 48 select(4, [3], [], [], {0, 200000}) = 0 (Timeout) sendto(3, "\6\1\0\0\0\16\2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 48, 0, {sa_family=AF_INET, sin_port=htons(323), sin_addr=inet_addr("127.0.0.1")}, 16) = 48 select(4, [3], [], [], {0, 400000}) = 0 (Timeout) fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 1), ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3fc15e8000 write(1, "506 Cannot talk to daemon\n", 26506 Cannot talk to daemon ) = 26 close(3) = 0 exit_group(1) = ? +++ exited with 1 +++ $

Hi Sandeep,

Please check if 'cmdport 0' option present in your /etc/chrony.conf file and uncomment it.

Regards,

Peter