munmap() should not fail when mm_struct.map_count temporarily reaches max_map_count

Solution Verified - Updated -

Issue

  • The munmap() syscall fails even  if the number of VMAs of a process (mm_struct.map_count) does not yet  reach the upper limit (sysctl_max_map_count).  The munmap() fails with ENOMEM if the number of VMAs equals the "limit - 1"  and it does not unmap a whole VMA but a part of a VMA.

  • In that case, the number of VMAs temporarily reaches the limit during munmap() and it backs to below the limit at the end of munmap(). So it
      should not fail.

Environment

  • Red Hat Enterprise Linux 5 Update 4
  • Red Hat Enterprise Linux 4 Update 8
  • ia64

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase of over 48,000 articles and solutions.

Current Customers and Partners

Log in for full access

Log In
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.