memset()'ing a large value can hang RHEL 5

Issue

We found our C program can hang on RHEL 5.5. But it's ok on RHEL 4.x.  The reproducer C program is as follows:

  #include <stdio.h>
  #include <stdlib.h>
  #include <string.h>
  #include <unistd.h>

  int main()
  {
      long sz = 4.1*1024*1024*1024;
      char *p = (char*)malloc(sz);
      memset(p, 0, sz);
      fork();
      return 0;
  }

If you save it to a file fork.c and compile with

     /usr/bin/g++  -pg fork.c

and run it on RHEL 5.5, you will see that the process will hang there.
On RHEL 4.x, it has no such issue.

1. But if we compile it without "-pg" option to /usr/bin/g++, then run it on RHEL5.5, the process will not hang.

2. If we replace the following line in fork.

            long sz = 4.1*1024*1024*1024;
   
        with the following line
   
   
   
            long sz = 4*1024*1024*1024;
   

     and then compile it with "/usr/bin/g++ -pg fork.c" and run it on
     RHEL 5.5, it will not hang on RHEL 5.5 any more.

Is this a kernel bug or a GCC bug?