(RHEL 4.8) java Websphere application giving glibc double free or corruption error

Solution Verified - Updated -

Issue

  • Customer has opened a case in regards to a websphere application crash on their server. The customer (Cisco) reached out to IBM for a core analysis and their development team is stating that the issue lies with our libc.so library. The vendor has provided there GDB analysis in a text file and I have it hosted.
    An admin opened a case INC000022249012 with me originally regarding memory usage issues, but then they also had an issue where their IBM Websphere would cause glibc to give a double free or corruption error.

  • I informed the admin that this is usually more likely an issue glibc detects memory issue (the way in which the application is using allocated memory, and not a memory hardware problem) when running 3rd party applications, and it tells the application to abort to safeguard the system, along with printing the above error message to the console.

He supplied the analysis IBM had done on the application cores:

Problem started when our WebSphere java process crashing with the following error.
*** glibc detected *** double free or corruption (out): 0x8170da98 *** 
The process generated a system core when the error occurred.

We had IBM analyze the system core.
  Our development team analyzed the core dump and found the server      

crashed on OS library:                                                                                                                     
(ip)=0085E8C5  at  libc.so.6 + 000278C5                                 
unframing (new_ebp=71F7ED54 new_eip=00860329)                           
(ip)=00860329  at  libc.so.6 + 00029329                                 
unframing (new_ebp=71F7EE54 new_eip=00892DCA)                           
(ip)=00892DCA  at  libc.so.6 + 0005BDCA                                 
unframing (new_ebp=71F7EECC new_eip=00898F5D)                           
(ip)=00898F5D  at  libc.so.6 + 00061F5D                                 
unframing (new_ebp=71F7EFA4 new_eip=00899F57)                           
(ip)=00899F57  at  libc.so.6 + 00062F57                                 
unframing (new_ebp=71F7EFD8 new_eip=0089BDC9)                           
(ip)=0089BDC9  at  libc.so.6 + 00064DC9                                 
unframing (new_ebp=71F7EFF8 new_eip=0804F656) >> libc.so libraries      

 The attachment has the full analysis from IBM.
The dumps are located on cepapp-nprd1-17  in /opt/shared/software/common/PMR/glibc/33120.49R.000.tar.gz
-----

I'll attach a sosreport, the application cores, and IBM's analysis.  I'd like some help reviewing these.

Environment

  • Red Hat Enterprise Linux 4.8
  • IBM WebSphere
  • gcc-java-3.4.6-11-x86_64 Wed 10 Nov 2010 05:10:22 PM CST
  • java-1.4.2-gcj-compat-1.4.2.0-27jpp-noarch Mon 06 Oct 2008 02:50:09 PM CDT
  • java-1.5.0-sun-1.5.0.20-1jpp.1.el4-i586 Wed 10 Nov 2010 05:27:00 PM CST

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.