named crashed with SIGSEGV

Solution In Progress - Updated -

Environment

  • Red Hat Enterprise Linux 6.1
  • kernel 2.6.32-279.5.1.el6.x86_64
  • bind-9.8.2-0.10.rc1.el6_3.5.x86_64

Issue

  • named crashed with SIGSEGV and dumped core file with following syslog entry:
Jun 12 01:15:31 hostnameA kernel: named[1702] general protection ip:7fca30b432a5 sp:7fca2d21acd0 error:0 in libisc.so.83.0.3[7fca30b25000+55000]
  • Stack trace shows:
Core was generated by `/usr/sbin/named -u named -t /var/named/chroot'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007fca30b432a5 in isc___mem_put (ctx0=0xdededededededede, ptr=0x7fca2a34cdd0, size=16, file=0x7fca31aedbda "resolver.c", line=8154) at mem.c:1319
1319        REQUIRE(VALID_CONTEXT(ctx));
(gdb) bt
#0  0x00007fca30b432a5 in isc___mem_put (ctx0=0xdededededededede, ptr=0x7fca2a34cdd0, size=16, file=0x7fca31aedbda "resolver.c", line=8154) at mem.c:1319
#1  0x00007fca31a65450 in dns_resolver_destroyfetch (fetchp=0x7fca2d21ad78) at resolver.c:8154
#2  0x00007fca3218d1f2 in query_resume (task=<value optimized out>, event=0x7fca15035980) at query.c:3636
#3  0x00007fca30b522f8 in dispatch (uap=0x7fca320fc010) at task.c:1012
#4  run (uap=0x7fca320fc010) at task.c:1157
#5  0x00007fca30507851 in start_thread (arg=0x7fca2d21b700) at pthread_create.c:301
#6  0x00007fca2fa6b11d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115
(gdb) info threads
  7 Thread 0x7fca321367c0 (LWP 1699)  0x00007fca2f9b5c54 in do_sigsuspend (set=<value optimized out>) at ../sysdeps/unix/sysv/linux/sigsuspend.c:63
  6 Thread 0x7fca2dc1c700 (LWP 1701)  0x00007fca31a0f34c in activeempty (search=0x7fca2dc1a0c0, chain=0x7fca2dc19750, name=0x7fc9f540d5c8) at rbtdb.c:2988
  5 Thread 0x7fca2e61d700 (LWP 1700)  0x00007fca3050eb3d in sendmsg () at ../sysdeps/unix/syscall-template.S:82
  4 Thread 0x7fca2c81a700 (LWP 1703)  0x00007fca2fa208ed in __tzfile_compute (timer=1370967331, use_localtime=<value optimized out>, 
    leap_correct=<value optimized out>, leap_hit=0x7fca2c817c9c, tp=0x7fca2fd153e0) at tzfile.c:830
  3 Thread 0x7fca2be19700 (LWP 1704)  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:216
  2 Thread 0x7fca2b418700 (LWP 1705)  0x00007fca2fa6b713 in epoll_wait () at ../sysdeps/unix/syscall-template.S:82
* 1 Thread 0x7fca2d21b700 (LWP 1702)  0x00007fca30b432a5 in isc___mem_put (ctx0=0xdededededededede, ptr=0x7fca2a34cdd0, size=16, 
    file=0x7fca31aedbda "resolver.c", line=8154) at mem.c:1319
(gdb)
  • The value of ctx0 is not the valid address since it has 0xdede...
(gdb) frame 1
#1  0x00007fca31a65450 in dns_resolver_destroyfetch (fetchp=0x7fca2d21ad78)
    at resolver.c:8154
8154        isc_mem_put(res->mctx, fetch, sizeof(*fetch));


ctx0 comes from res->mctx.

(gdb) print res->mctx
$38 = (isc_mem_t *) 0xdededededededede

(gdb) x/100x res
0x7fca2a3404a8: 0x2a340320  0x00007fca  0xdededede  0xdededede
0x7fca2a3404b8: 0xdededede  0xdededede  0xdededede  0xdededede
0x7fca2a3404c8: 0xdededede  0xdededede  0xdededede  0xdededede
0x7fca2a3404d8: 0xdededede  0xdededede  0xdededede  0xdededede
0x7fca2a3404e8: 0xdededede  0xdededede  0xdededede  0xdededede
0x7fca2a3404f8: 0xdededede  0xdededede  0xdededede  0xdededede

So, 'res' is freed already.

Resolution

  • An errata was released to fix this bug: RHBA-2014:1373
  • A reproducer helps to address this problem.
  • Turning on debug log with level 5 also produces additional info.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments