page_fault while low on memory results in stack overflow with btrfs filesystem

Solution Verified - Updated -

Issue

  • page_fault while low on memory results in stack overflow with btrfs filesystem
  • System crashed with below messages:
[   42.432578] kernel tried to execute NX-protected page - exploit attempt? (uid: 0)
[   42.433307] BUG: unable to handle kernel paging request at ffff88028b78bfd8
[   42.433918] IP: [] 0xffff88028b78bfd7
[   42.434650] PGD 1f9f067 PUD 1fa2067 PMD 28b7be063 PTE 800000028b78b163
[   42.435249] Thread overran stack, or stack corrupted
[   42.435659] Oops: 0011 [#1] SMP 
  • Server crashed while running container with memory limit and btrfs as backend filesystem.

Environment

  • Red Hat Enterprise Linux 7
  • Docker engine 1.12.6
  • BTRFS storage driver
  • cgroup

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content