httpd child process shutdown stalled in http2 when using mpm_worker
Issue
- If we shutdown httpd or do a graceful restart/reload with mod_http2 configured in combination with mpm_worker, then we can see the main child process hung in join_workers like below:
Thread 1 (Thread 0x7fe56ec3f540 (LWP 483878) "httpd"):
#0 0x00007fe56ea868ba in __futex_abstimed_wait_common () from target:/lib64/libc.so.6
#1 0x00007fe56ea8b2b3 in __pthread_clockjoin_ex () from target:/lib64/libc.so.6
#2 0x00007fe56ec792ef in apr_thread_join () from target:/lib64/libapr-1.so.0
#3 0x00007fe56ec32f59 in join_workers () from target:/etc/httpd/modules/mod_mpm_worker.so
#4 0x00007fe56ec34a88 in child_main () from target:/etc/httpd/modules/mod_mpm_worker.so
#5 0x00007fe56ec34e0a in make_child () from target:/etc/httpd/modules/mod_mpm_worker.so
#6 0x00007fe56ec34eb1 in startup_children () from target:/etc/httpd/modules/mod_mpm_worker.so
#7 0x00007fe56ec3599c in worker_run () from target:/etc/httpd/modules/mod_mpm_worker.so
#8 0x0000557c346f06d8 in ap_run_mpm ()
#9 0x0000557c346de64f in main ()
- This is waiting for child threads to complete, which pending ones are process http2 connections in calls like below:
Thread 182 (Thread 0x7fe345fd3640 (LWP 485407) "httpd"):
#0 0x00007fe56eb0de3e in epoll_wait () from target:/lib64/libc.so.6
#1 0x00007fe56ec77310 in impl_pollset_poll.lto_priv () from target:/lib64/libapr-1.so.0
#2 0x00007fe56e355af9 in h2_mplx_c1_poll.constprop () from target:/etc/httpd/modules/mod_http2.so
#3 0x00007fe56e35b0c2 in h2_c1_run.isra () from target:/etc/httpd/modules/mod_http2.so
#4 0x00007fe56e340ad0 in h2_c1_hook_process_connection.lto_priv () from target:/etc/httpd/modules/mod_http2.so
#5 0x0000557c346efda8 in ap_run_process_connection ()
#6 0x00007fe56ec344dc in worker_thread () from target:/etc/httpd/modules/mod_mpm_worker.so
#7 0x00007fe56ea897e2 in start_thread () from target:/lib64/libc.so.6
#8 0x00007fe56eb0e800 in clone3 () from target:/lib64/libc.so.6
Environment
- Red Hat Enterprise Linux (RHEL)
- JBoss Core Services (JBCS)
- Apache httpd
- mod_http2
- mpm_worker
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.