Apache httpd is frequently crashing when sending a client cert to backend server using mod_proxy with Two-way SSL
Issue
- The httpd is configured as a reverse proxy server which connects with the backend server with 2-way ssl.When it tries to send a client certificate, it exits with abort:
#0 0x00007f2774430387 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
#1 0x00007f2774431a78 in __GI_abort () at abort.c:90
#2 0x00007f2774472ed7 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7f2774585350 "*** Error in `%s': %s: 0x%s ***\n")
at ../sysdeps/unix/sysv/linux/libc_fatal.c:196
#3 0x00007f277447b299 in malloc_printerr (ar_ptr=0x7f27747c1760 <main_arena>, ptr=<optimized out>,
str=0x7f2774585410 "double free or corruption (fasttop)", action=3) at malloc.c:4967
#4 _int_free (av=0x7f27747c1760 <main_arena>, p=<optimized out>, have_lock=0) at malloc.c:3843
#5 0x00007f276942580d in CRYPTO_free (str=0x55e8a8924950) at mem.c:434
#6 0x00007f27694eb556 in EVP_PKEY_free (x=<optimized out>) at p_lib.c:406
#7 0x00007f276983f92d in ssl3_send_client_certificate (s=s@entry=0x7f2754024050) at s3_clnt.c:3469
#8 0x00007f276983fecf in ssl3_connect (s=0x7f2754024050) at s3_clnt.c:421
#9 0x00007f2769849e7e in ssl23_get_server_hello (s=0x7f2754024050) at s23_clnt.c:799
#10 ssl23_connect (s=0x7f2754024050) at s23_clnt.c:228
#11 0x00007f2769aa10d8 in ssl_io_filter_handshake (filter_ctx=filter_ctx@entry=0x7f27480091d0) at ssl_engine_io.c:1086
#12 0x00007f2769aa22f4 in ssl_io_filter_output (f=0x7f2748009228, bb=0x7f2754004840) at ssl_engine_io.c:1616
#13 0x00007f2769a9ef9a in ssl_io_filter_coalesce (f=0x7f2748009200, bb=0x7f2754004840) at ssl_engine_io.c:1584
#14 0x00007f276b92c877 in ap_proxy_pass_brigade (bucket_alloc=bucket_alloc@entry=0x7f27540008e8, r=r@entry=0x7f2754002970,
p_conn=p_conn@entry=0x7f2754006990, origin=origin@entry=0x7f2748008bb0, bb=bb@entry=0x7f2754004840, flush=flush@entry=1)
at proxy_util.c:3556
#15 0x00007f276a0d4061 in stream_reqbody_cl (old_cl_val=<optimized out>, input_brigade=0x7f2754010b38, header_brigade=0x7f2754004840,
origin=0x7f2748008bb0, p_conn=0x7f2754006990, r=0x7f2754002970, p=0x7f27540028f8) at mod_proxy_http.c:482
#16 ap_proxy_http_request (server_portstr=0x7f276532aa20 "", url=<optimized out>, uri=0x7f27540047a0, conf=0x55e8a88e27c0,
worker=<optimized out>, p_conn=0x7f2754006990, r=0x7f2754002970, p=0x7f27540028f8) at mod_proxy_http.c:933
#17 proxy_http_handler (r=<optimized out>, worker=<optimized out>, conf=0x55e8a88e27c0,
url=0x7f2754004546 "https://localhost:10443/test/", proxyname=0x0, proxyport=0) at mod_proxy_http.c:2022
#18 0x00007f276b923f34 in proxy_run_scheme_handler (r=r@entry=0x7f2754002970, worker=0x55e8a88e6ef8, conf=conf@entry=0x55e8a88e27c0,
url=0x7f2754004546 "https://localhost:10443/test/", proxyhost=proxyhost@entry=0x0, proxyport=proxyport@entry=0)
at mod_proxy.c:2746
#19 0x00007f276b924e1d in proxy_handler (r=0x7f2754002970) at mod_proxy.c:1125
#20 0x000055e8a7e03f40 in ap_run_handler (r=r@entry=0x7f2754002970) at config.c:169
#21 0x000055e8a7e04489 in ap_invoke_handler (r=r@entry=0x7f2754002970) at config.c:439
#22 0x000055e8a7e190da in ap_process_async_request (r=r@entry=0x7f2754002970) at http_request.c:339
#23 0x000055e8a7e193c2 in ap_process_request (r=r@entry=0x7f2754002970) at http_request.c:374
#24 0x000055e8a7e15552 in ap_process_http_sync_connection (c=0x7f2758004e18) at http_core.c:190
#25 ap_process_http_connection (c=0x7f2758004e18) at http_core.c:231
#26 0x000055e8a7e0d570 in ap_run_process_connection (c=c@entry=0x7f2758004e18) at connection.c:41
#27 0x000055e8a7e0d988 in ap_process_connection (c=c@entry=0x7f2758004e18, csd=csd@entry=0x7f2758004c00) at connection.c:212
#28 0x00007f276bb3b76b in process_socket (bucket_alloc=0x7f27540008e8, my_thread_num=2, my_child_num=0, sock=0x7f2758004c00,
p=0x7f2758004b78, thd=0x55e8a8943d00) at worker.c:620
#29 worker_thread (thd=0x55e8a8943d00, dummy=<optimized out>) at worker.c:979
#30 0x00007f27749d3ea5 in start_thread (arg=0x7f276532b700) at pthread_create.c:307
#31 0x00007f27744f88dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Environment
- Red Hat Enterprise Linux (RHEL) 7.8
- httpd-2.4.6-95
- mod_ssl-2.4.6-95
- openssl-1.0.2k-19
- openssl-libs-1.0.2k-19
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.