memset()'ing a large value can hang RHEL 5

Solution Verified - Updated -

Issue

We found our C program can hang on RHEL 5.5. But it's ok on RHEL 4.x.  The reproducer C program is as follows:

  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <unistd.h>

  int main()
  {
      long sz = 4.1*1024*1024*1024;
      char *p = (char*)malloc(sz);
      memset(p, 0, sz);
      fork();
      return 0;
  }

If you save it to a file fork.c and compile with

     /usr/bin/g++  -pg fork.c

and run it on RHEL 5.5, you will see that the process will hang there.
On RHEL 4.x, it has no such issue.

  1. But if we compile it without "-pg" option to /usr/bin/g++, then run it on RHEL5.5, the process will not hang.
  2. If we replace the following line in fork.

             long sz = 4.1*1024*1024*1024;
    
         with the following line
    
    
    
             long sz = 4*1024*1024*1024;
    

     and then compile it with "/usr/bin/g++ -pg fork.c" and run it on
     RHEL 5.5, it will not hang on RHEL 5.5 any more.

Is this a kernel bug or a GCC bug?

Environment

  • Hangs in Red Hat Enterprise Linux (RHEL) 5.5

  • Works in RHEL 4

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.