httpd child process destruction stalls in wsgi_python_child_cleanup

Solution Unverified - Updated -

Issue

  • After some time, httpd becomes unresponsive and begins to continuously log the following:
[mpm_event:error] [pid 1577:tid 1577] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
  • All child processes are created and they are all persisting in wsgi_python_child_cleanup:
    PID: 655889
#0  0x000003ffa649ab26 in __futex_abstimed_wait_cancelable64 () from target:/lib64/libc.so.6
#1  0x000003ffa64a63d8 in do_futex_wait.constprop () from target:/lib64/libc.so.6
#2  0x000003ffa64a6478 in __new_sem_wait_slow64.constprop.0 () from target:/lib64/libc.so.6
#3  0x000003ffa1baafba in PyThread_acquire_lock_timed () from target:/lib64/libpython3.9.so.1.0
#4  0x000003ffa1c65426 in acquire_timed () from target:/lib64/libpython3.9.so.1.0
#5  0x000003ffa1c651be in lock_PyThread_acquire_lock () from target:/lib64/libpython3.9.so.1.0
#6  0x000003ffa1bd3132 in method_vectorcall_VARARGS_KEYWORDS () from target:/lib64/libpython3.9.so.1.0
#7  0x000003ffa1bc0e8c in _PyEval_EvalFrameDefault () from target:/lib64/libpython3.9.so.1.0
#8  0x000003ffa1bd1e24 in function_code_fastcall () from target:/lib64/libpython3.9.so.1.0
#9  0x000003ffa1bd19e8 in _PyFunction_Vectorcall () from target:/lib64/libpython3.9.so.1.0
#10 0x000003ffa2011f48 in ShutdownInterpreter_call () from target:/etc/httpd/modules/mod_wsgi_python3.so
#11 0x000003ffa1bcb832 in _PyObject_MakeTpCall () from target:/lib64/libpython3.9.so.1.0
#12 0x000003ffa1be8604 in PyObject_VectorcallMethod () from target:/lib64/libpython3.9.so.1.0
#13 0x000003ffa1ca13a4 in wait_for_thread_shutdown () from target:/lib64/libpython3.9.so.1.0
#14 0x000003ffa1b9e072 in Py_EndInterpreter () from target:/lib64/libpython3.9.so.1.0
#15 0x000003ffa2010a7e in Interpreter_dealloc () from target:/etc/httpd/modules/mod_wsgi_python3.so
#16 0x000003ffa1bbc1d8 in free_keys_object () from target:/lib64/libpython3.9.so.1.0
#17 0x000003ffa2019e4c in wsgi_python_child_cleanup () from target:/etc/httpd/modules/mod_wsgi_python3.so
#18 0x000003ffa66ad052 in apr_pool_destroy () from target:/lib64/libapr-1.so.0
#19 0x000003ffa3e863f8 in clean_child_exit () from target:/etc/httpd/modules/mod_mpm_event.so
#20 0x000003ffa3e89180 in child_main () from target:/etc/httpd/modules/mod_mpm_event.so
#21 0x000003ffa3e89744 in make_child () from target:/etc/httpd/modules/mod_mpm_event.so
#22 0x000003ffa3e89806 in startup_children () from target:/etc/httpd/modules/mod_mpm_event.so
#23 0x000003ffa3e8a8f4 in event_run () from target:/etc/httpd/modules/mod_mpm_event.so
#24 0x000002aa39cae3ca in ap_run_mpm ()
#25 0x000002aa39c996d8 in main ()

Environment

  • Red Hat Enterprise Linux (RHEL)
  • Apache httpd
  • python3-mod_wsgi

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content