Automount segfaults in libnsss_sss.so.2.

Solution In Progress - Updated -

Environment

  • Red Hat Enterprise Linux 6.4
  • autofs-5.0.5-74.el6_4.x86_64

Issue

Automount segfaults in libnsss_sss.so.2.

automount[17714]: segfault at 7fffe4053000 ip 00007fffe455007e sp 00007fffccb44a08 error 4 in libnss_sss.so.2[7fffe454b000+7000]

Resolution

Diagnostic Steps

  • Analyze automount application core:
Core was generated by `automount -t 300 -O tcp -DNC1= -DNCA= -DNCB= -DOSREL=6 -DARCH=x86_64 -DOSNAME=r'.
Program terminated with signal 11, Segmentation fault.
#0  sss_nss_check_header (ctx=0x7fffe47524a0) at /usr/include/bits/string3.h:52
52      /usr/include/bits/string3.h: No such file or directory.
        in /usr/include/bits/string3.h
(gdb) bt
#0  sss_nss_check_header (ctx=0x7fffe47524a0) at /usr/include/bits/string3.h:52
#1  0x00007fffe45501ce in sss_nss_mc_get_ctx (name=0x7fffe4550ef2 "group", ctx=0x7fffe47524a0) at src/sss_client/nss_mc_common.c:105
#2  0x00007fffe4550a01 in sss_nss_mc_getgrgid (gid=12676, result=0x7fffccb44cc0, buffer=0x7fffa80008c0 "pe-muppet", buflen=1024) at src/sss_client/nss_mc_group.c:183
#3  0x00007fffe454ec4a in _nss_sss_getgrgid_r (gid=12676, result=0x7fffccb44cc0, buffer=0x7fffa80008c0 "pe-muppet", buflen=1024, errnop=0x7fffccb476a8) at src/sss_client/nss_group.c:455
#4  0x00007ffff6a6e4dd in __getgrgid_r (gid=12676, resbuf=0x7fffccb44cc0, buffer=0x7fffa80008c0 "pe-muppet", buflen=1024, result=0x7fffccb44ce0) at ../nss/getXXbyYY_r.c:253
#5  0x00007ffff7fdef3e in set_tsd_user_vars (logopt=0, uid=<value optimized out>, gid=12676) at mounts.c:1214
#6  0x00007ffff7fd17a8 in do_mount_indirect (arg=<value optimized out>) at indirect.c:810
#7  0x00007ffff7b8c851 in start_thread (arg=0x7fffccb47700) at pthread_create.c:301
#8  0x00007ffff6aac90d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

(gdb) disass sss_nss_check_header,0x00007fffe4550084
Dump of assembler code from 0x7fffe4550070 to 0x7fffe4550084:
   0x00007fffe4550070 <sss_nss_check_header+0>: mov    0x10(%rdi),%rax
   0x00007fffe4550074 <sss_nss_check_header+4>: lea    -0x48(%rsp),%rdx
   0x00007fffe4550079 <sss_nss_check_header+9>: mov    $0x5,%ecx
=> 0x00007fffe455007e <sss_nss_check_header+14>:        mov    (%rax),%rsi
   0x00007fffe4550081 <sss_nss_check_header+17>:        mov    %rsi,(%rdx)
End of assembler dump.

So we can see we crashed while dereferencing %rax.

(gdb) i r rax
rax            0x7fffe4053000   140737018933248
(gdb) x 0x7fffe4053000
0x7fffe4053000: Cannot access memory at address 0x7fffe4053000

%rdi has to be our first argument to the function so we can get at that from the frame above.

(gdb) p ctx
$1 = (struct sss_cli_mc_ctx *) 0x7fffe47524a0
(gdb) ptype struct sss_cli_mc_ctx
type = struct sss_cli_mc_ctx {
    _Bool initialized;
    int fd;
    uint32_t seed;
    void *mmap_base;
    size_t mmap_size;
    uint8_t *data_table;
    uint32_t dt_size;
    uint32_t *hash_table;
    uint32_t ht_size;
}
(gdb) p &((struct sss_cli_mc_ctx *)0x0)->mmap_base
$2 = (void **) 0x10
(gdb) p ctx->mmap_base 
$3 = (void *) 0x7fffe4053000

This is potentially a race with thread 18.
(gdb) t 18
[Switching to thread 18 (Thread 0x7fffcc844700 (LWP 17713))]#0  0x00007ffff7b935ad in close () at ../sysdeps/unix/syscall-template.S:82
82      T_PSEUDO (SYSCALL_SYMBOL, SYSCALL_NAME, SYSCALL_NARGS)
(gdb) bt
#0  0x00007ffff7b935ad in close () at ../sysdeps/unix/syscall-template.S:82
#1  0x00007fffe455029c in sss_nss_mc_get_ctx (name=0x7fffe4550ef2 "group", ctx=0x7fffe47524a0) at src/sss_client/nss_mc_common.c:164
#2  0x00007fffe4550a01 in sss_nss_mc_getgrgid (gid=12676, result=0x7fffcc841cc0, buffer=0x7fff980008c0 "pe-muppet", buflen=1024) at src/sss_client/nss_mc_group.c:183
#3  0x00007fffe454ec4a in _nss_sss_getgrgid_r (gid=12676, result=0x7fffcc841cc0, buffer=0x7fff980008c0 "pe-muppet", buflen=1024, errnop=0x7fffcc8446a8) at src/sss_client/nss_group.c:455
#4  0x00007ffff6a6e4dd in __getgrgid_r (gid=12676, resbuf=0x7fffcc841cc0, buffer=0x7fff980008c0 "pe-muppet", buflen=1024, result=0x7fffcc841ce0) at ../nss/getXXbyYY_r.c:253
#5  0x00007ffff7fdef3e in set_tsd_user_vars (logopt=0, uid=<value optimized out>, gid=12676) at mounts.c:1214
#6  0x00007ffff7fd17a8 in do_mount_indirect (arg=<value optimized out>) at indirect.c:810
#7  0x00007ffff7b8c851 in start_thread (arg=0x7fffcc844700) at pthread_create.c:301
#8  0x00007ffff6aac90d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115
  • Customer is running a patched version of sssd-client (sssd-client-1.9.2-129.el6_5.5)

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.