Virtual file system (VFS) native memory leak in JBoss EAP 5

Issue

• Process size grows very fast when JBoss is processing and caching annotations, which utilizes VFS a lot.
• I have noticed a huge memory consumption by our JBoss EAP 5 instances. In particular I have noticed that a Jboss instance that starts with the memory java parameters set to “-Xmx=4GB” , “- Xms=4GB “ , “-XX:PermSize=512M” and “-XX:MaxPermSize=512M”, gets a memory footprint on Operative System roughly 7GB.
• The memory footprint for JBoss EAP 5 seems to be significantly higher than the jvm heap configuration and even after taking into account the thread stack and PermGen, the memory consumed is much higher than we envisaged.
• JBoss is configured to use 8 GiB of Heap. It's also configured to have a maximum of 200 threads, by which default take up a maximum of 512KB of stack. However, it is getting 11 GB of memory being used by JBoss EAP 5, which is very close the maximum system memory and eventually the OS kills it to regain the memory.
  Also, from GC logs, the Heap is not full and only taking up to ~2 GB, so not even close to the 8GiB of memory.