Stack Guard Page Circumvention Affecting Multiple Packages

Public Date: June 19, 2017, 12:00 am
Updated -
Resolved Status
Important Impact

To get released updates to address this issue use the Resolve tab.

Red Hat Product Security has been made aware of a vulnerability affecting Linux systems that allows for privilege escalation. This vulnerability has been assigned two CVE names, CVE-2017-1000364 for the Linux kernel and CVE-2017-1000366 for glibc. This issue was publicly disclosed on June 19th, 2017 and has been rated as Important.

Background Information

Qualys has released a security advisory showing practical methods for circumventing an exploit protection mechanism known as the "stack guard page".  A flaw was found in the way memory was being allocated on the stack for user-space binaries. If heap (or a different memory region) and stack memory regions were adjacent to each other, an attacker could use this flaw to jump over the stack guard page, causing controlled memory corruption on the process stack or the adjacent memory region, thus increasing their privileges on the system.

What is a stack guard page?
Stack guard page was added to the Linux kernel as a protection against stack overflow attacks and primarily to fix CVE-2010-2240. Access to the stack guard page triggers a trap, so it serves as a divider between a stack memory region and other memory regions in the process address space so that sequential stack access cannot be fluently transformed into access to another memory region adjacent to the stack (and vice versa).
What is the flaw?

The stack guard page was originally meant as a protection against sequential memory access. The reporter (Qualys) found ways to re-introduce CVE-2010-2240 in generic binaries not by causing sequential stack overflow (and thus running into the stack guard page), but by leveraging certain constructs in stack memory allocation, as performed by common binaries, to "jump" over the stack guard page and again be able to access memory in the adjacent memory region without causing an invalid access. This results in overlapping stack with another memory region (usually heap) and thus stack memory access is reflected into the heap and vice versa.

What are the fixes? 
To avoid stack guard page jumping, every stack allocation primitive needs to implement freshly allocated memory probing with the stack guard gap size granularity. The existing gcc -fstack-check implementation aims to do exactly that, but currently it is not working correctly. Before the gcc -fstack-check implementation is fixed and all of the exposed binaries are rebuilt, we have a combination of kernel and glibc mitigations that addresses all known reporter-provided exploits available:

  • kernel fix for CVE-2017-1000364 increases the stack guard gap size from one page to 1 MiB to make successful exploitation of this issue more difficult
  • glibc fix for CVE-2017-1000366 blocks the use of LD_LIBRARY_PATH in AT_SECURE=1 programs. This acts as a form of input sanitization for the internal DSO search path construction, which would otherwise perform a very large alloca() with a partially-filled buffer. As additional hardening, the amount of heap and stack allocations performed by the processing of LD_AUDIT, LD_PRELOAD, and LD_HWCAP_MASK has been reduced. Previously, they could be used to shape heap and stack allocations and we believe that, after these changes, this has become more difficult.

What are the attack vectors?

This vulnerability is in fact a class of attacks against a category of memory protections and mitigation mechanisms.  The attacks can be performed through any of the traditionally known attack vectors, however the CVSS scores provided by Red Hat are a worse-case scenario.  We expect the most common form of attack will be attackers with a local account, able to control the conditions in which processes are started.  Network-facing processes may be affected, and Red Hat strongly recommends upgrading the kernel and glibc to minimise these attack vectors.

What are the side effects of these mitigation patches?

At the time of this writing there is a known issue introduced by the kernel patch which creates overlapping values in /proc/meminfo.  This should not affect the functional application of the system and protection provided by the kernel.   A patch to address this issue may be released at a later date.

Some processes which set threads stack guard size manually may not correctly handle the guard page size change and may need to be adjusted to the new changed size (See man pthread_attr_setguardsize) and correctly handle this changed condition.

Acknowledgement

Red Hat would like to thank Qualys Research Labs for reporting this flaw.

Successful exploitation of this vulnerability could allow an attacker to escalate privileges and potentially run malicious code.

Red Hat Product Security has rated this update as having a security impact of Important.    

Impacted Products

The following Red Hat product versions are impacted:

  • Red Hat Enterprise Linux 5
  • Red Hat Enterprise Linux 6
  • Red Hat Enterprise Linux 7
  • Red Hat Enterprise MRG 2.5
  • Red Hat Virtualization
  • RHEL Atomic Host

Diagnose your vulnerability


Determine if your system is vulnerable

Use the detection script below to determine if your system is currently vulnerable to this flaw.  To verify the legitimacy of the script, you can download the detached GPG signature as well.  The current version of the script is 1.1.

Download Detection Script

Take Action

All Red Hat customers running affected products are strongly recommended to update as soon as patches are available. Details about impacted packages are noted below. A system reboot is required in order for the kernel update to be applied.

A kpatch for customers running Red Hat Enterprise Linux 7.2 or greater is available. Please open a support case to gain access to the kpatch.

For more details about what a kpatch is: Is live kernel patching (kpatch) supported in RHEL 7?


ProductPackageAdvisory/Update
Red Hat Enterprise Linux 7kernelRHSA-2017:1484
Red Hat Enterprise Linux 7kernel-rtRHSA-2017:1616
Red Hat Enterprise Linux 7glibcRHSA-2017:1481
Red Hat Enterprise Linux 7.2 Extended Update Support**kernelRHSA-2017:1485
Red Hat Enterprise Linux 7.2 Extended Update Support**glibcRHSA-2017:1479
Red Hat Enterprise Linux 6kernelRHSA-2017:1486
Red Hat Enterprise Linux 6glibcRHSA-2017:1480
Red Hat Enterprise Linux 6.7 Extended Update Support**kernelRHSA-2017:1487
Red Hat Enterprise Linux 6.7 Extended Update Support**glibcRHSA-2017:1479
Red Hat Enterprise Linux 6.6 Advanced Update Support***kernelRHSA-2017:1488
Red Hat Enterprise Linux 6.6 Advanced Update Support***glibcRHSA-2017:1479
Red Hat Enterprise Linux 6.5 Advanced Update Support***kernelRHSA-2017:1489
Red Hat Enterprise Linux 6.5 Advanced Update Support***glibcRHSA-2017:1479
Red Hat Enterprise Linux 6.4 Advanced Update Support***kernelRHSA-2017:1490
Red Hat Enterprise Linux 6.4 Advanced Update Support***glibcRHSA-2017:1479
Red Hat Enterprise Linux 6.2 Advanced Update Support***kernelRHSA-2017:1491
Red Hat Enterprise Linux 5 ELS*kernelRHSA-2017:1482
Red Hat Enterprise Linux 5 ELS*glibcRHSA-2017:1479
Red Hat Enterprise Linux 5.9 Advanced Update Support***kernelRHSA-2017:1483
Red Hat Enterprise Linux 5.9 Advanced Update Support***glibcRHSA-2017:1479
RHEL Atomic HostkernelImages respun on 21June2017
Red Hat Enterprise MRG 2kernel-rtRHSA-2017:1647
Red Hat Virtualization Hypervisor (RHV-H) Image 3.6kernelRHEA-2017:1569
Red Hat Enterprise Virtualization (RHEV-H) Hypervisor 3.6kernelRHBA-2017:1568
Red Hat Enterprise Virtualization Manager (RHEV-M) Appliance 3.6kernelRHBA-2017:1571
Red Hat Virtualization Hypervisor (RHV-H) Image 4.1kernelRHBA-2017:1566
Red Hat Virtualization Manager (RHV-M) Appliance 4.1kernelRHEA-2017:1570


*An active ELS subscription is required for access to this patch.

Please contact Red Hat sales or your specific sales representative for more information if your account does not have an active ELS subscription.

**An active EUS subscription is required for access to this patch.

Please contact Red Hat sales or your specific sales representative for more information if your account does not have an active EUS subscription.

What is the Red Hat Enterprise Linux Extended Update Support Subscription?

***An active AUS subscription is required for access to this patch in RHEL AUS.

Ansible Playbook

An Ansible  playbook is available.

The playbook runs against a variable named HOSTS, and can be invoked as follows (assuming 'hostname' is defined in your inventory file): 

# ansible-playbook -e HOSTS=hostname cve-2017-1000366.yml

This playbook requires root privileges, so you may need to specify --become if it's not defined for 'hostname' in your inventory file.

Share

Red Hat Customer Portal Twitter Facebook

30 Comments

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

TC Newbie 5 points
19 June 2017 6:33 PM

The script needs to be fixed up. In the output I receive:

 Detected 'glibc' package is ''.

When I believe it should be:

Detected 'glibc' package is 'glibc-2.17-157.el7_3.1.x86_64'.

In which case, line 920 needs to be fixed up to have:

${BOLD}$vulnerable_package${RESET}.

or put an 's' at the end of the $installed_package part of the string...

ITS BR Hosting UNIX DHL's picture
19 June 2017 7:30 PM

Hi Ted,

On the referred line is missing a 's' in the variable name:

Original Line: echo -e "Detected 'glibc' package is '${BOLD}$installed_package${RESET}'."

Fix: echo -e "Detected 'glibc' package is '${BOLD}$installed_packages${RESET}'."

Output:

./cve-20171000366-1.sh

This script is primarily designed to detect CVE-1000366 on supported Red Hat Enterprise Linux systems and kernel packages. Result may be inaccurate for other RPM based systems.

Detected 'glibc' package is 'glibc-2.12-1.209.el6_9.1.x86_64'. Detected 'kernel' package is '2.6.32-696.3.1.el6.x86_64'.

Stanislav Kontar's picture Red Hat Community Member 37 points
20 June 2017 2:50 PM

Hi Ted. Thanks for reporting the issue. There were some last minute changes which were not properly tested. The fixed version of the script is uploaded now. We are sorry for the inconvenience.

BB Newbie 2 points
19 June 2017 8:49 PM

Mine wasn't detecting glibc as vulnerable. The part that strips the arch out was changing "glibc-2.5-123.el5_11.3" to "glibc-2.5-123.el5_11". I made these changes to fix it. (The diff includes the previously mentioned missing 's' in a variable name, too).

--- cve-2017-1000366_1.sh       2017-06-19 10:56:53.000000000 -0400
+++ cve-2017-1000366_1_bbeck.sh 2017-06-19 16:38:36.000000000 -0400
@@ -877,2 +877,3 @@
         for installed_package in "${installed_packages[@]}"; do
+            # strip arch from package name (as everything following the last dot)
             installed_package_without_arch="${installed_package%.*}"
@@ -893,4 +894,4 @@

-    # Get installed glibc packages
-    installed_packages=$( rpm -qa "glibc" )
+    # Get installed glibc packages, (include arch here so it can be safely/cleanly stripped later)
+    installed_packages=$( rpm -qa --qf "%{n}-%{v}-%{r}.%{arch}\n" "glibc" | sort | xargs )

@@ -919,3 +920,3 @@
     # Results
-    echo -e "Detected 'glibc' package is '${BOLD}$installed_package${RESET}'."
+    echo -e "Detected 'glibc' package is '${BOLD}$installed_packages${RESET}'."
     echo -e "Detected 'kernel' package is '${BOLD}$running_kernel${RESET}'."
Stanislav Kontar's picture Red Hat Community Member 37 points
20 June 2017 2:50 PM

Hi Brent. Thanks for reporting the issue and suggesting the patch. There were some last minute changes to the algorithm and we did not test them on RHEL5. Sorry about that. The fixed version of the script is uploaded now.

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
19 June 2017 11:33 PM 
A kpatch for customers running Red Hat Enterprise Linux 7.2 or greater is available. Please open a support case to gain access to the kpatch.

Is it right to assume that this will only partially solve/cover the problem as it is for live kernel patching and this issue covers kernel/glibc, unless kpatch now provides userspace live patching of glibc?

Cliff Perry's picture Red Hat Community Member 25 points
20 June 2017 10:54 AM

You are correct. Currently the kpatch solution is available for kernel packages only, and does not provide glibc fixes.

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
19 June 2017 11:35 PM 
The stack guard page was originally meant as a protection against sequential memory access.

Note, there are obvious edits to this document with the insertion of the words 'originally' and 'sequential' displaying with different fonts throughout.

Cliff Perry's picture Red Hat Community Member 25 points
20 June 2017 12:24 PM

Thanks for noticing. I fixed those that I saw. Regards,

Wade Mealing's picture Red Hat Pro 549 points
21 June 2017 1:17 AM

That will teach me for editing on a system with very few fonts ;) Most of the fonts fell back to a single font and I didnt notice.

Oscar Juarez Perez's picture Community Member 90 points
20 June 2017 8:18 AM

When will be the "patch" of the script uploaded or updated to version 1.1?

Stanislav Kontar's picture Red Hat Community Member 37 points
20 June 2017 2:52 PM

Hi Oscar. I am sorry it took so long. The fixed version of the script is available now.

TG Red Hat Community Member 35 points
20 June 2017 3:14 PM

It would really help to describe the attack vector here for customers. The https://access.redhat.com/security/cve/cve-2017-1000364 says LOCAL and COMPLEX but some plain english description of this on this page would be VERY helpful if it can be described here.

Wade Mealing's picture Red Hat Pro 549 points
21 June 2017 1:10 AM

Gday Tom,

Updated the article to include the attack vector header. Is that what you were after ?

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
21 June 2017 6:40 AM

Some feedback about the posted Ansible playbooks... it would be nice if this playbook links instead linked to a git repo that could be viewed/browsed (eg. github/gitlab).

Downloading and extracting a tarball to view some basic yaml is tedious. If it's viewable online it can be used to quickly assist admins in determining the complexity of the fix (ie. quick, technical, coded summary).

LB Newbie 11 points
21 June 2017 5:14 PM

The script didn't work for me.

Update I change the permissions to file and it worked.

Stanislav Kontar's picture Red Hat Community Member 37 points
22 June 2017 9:55 AM

Hi Laval. Can you give a bit more detail about how did the script fail? Does it print any error messages?

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
22 June 2017 11:12 PM

I have deployed this to several hosts and it all looks OK, but may have found an issue with Oracle DB Clusterware not starting with this specific kernel. When the kernel update was backed out/reverted to previous kernel (two alternate versions tried) the services started fine again. Have relinked Oracle binaries with new kernel and the issue persists.

Clusterware starts without issue on the following kernels (kernels that were reverted to)

kernel-3.10.0-514.16.1.el7.x86_64
kernel-3.10.0-514.21.1.el7.x86_64

As soon as we moved to the version listed with this errata, the issue occurs (with very little relevant log output).

kernel-3.10.0-514.21.2.el7.x86_64

Clusterware version is 12.1.0.2. Have raised this with Oracle. If anyone else can drop a message if they have any similar issues it would be appreciated. Not 100% it's related at the moment, but isn't as expected.

Cliff Perry's picture Red Hat Community Member 25 points
23 June 2017 10:39 AM

I am aware of one ongoing issue which Red Hat Kernel Engineering is tracking. That is the following bugzilla:

Bug 1463241 - rlimit_stack problems after update to 3.10.0-514.21.2.el7 - https://bugzilla.redhat.com/show_bug.cgi?id=1463241

Please open a support ticket with Red Hat to investigate your specific circumstances.

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
25 June 2017 10:23 PM

This Tomcat issue can be added to the list too:

https://access.redhat.com/solutions/3091371

Interesting discussion here about the upstream patch differing from that used by the distros:

http://seclists.org/oss-sec/2017/q2/568

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
26 June 2017 7:21 AM

We just had the exact JVM issue (as above link describes) occur in our application platform

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGBUS (0x7) at pc=0x00007f6604198461, pid=5497, tid=140076329084736
#
# Problematic frame:
# j  java.lang.Object.<clinit>()V+0
#
# Core dump written.
Akemi Yagi's picture Guru 4327 points
26 June 2017 10:02 PM

The original post in the referenced thread (subject: stackguard fix in Red Hat and Ubuntu kernels) has a reproducer as an attachment sk.c. When built and run on RHEL 6 and RHEL 7 (both the latest kernel), it produced "Bus error (core dumped)".

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
26 June 2017 11:53 PM

Have seen the reproducer.

This post mentions that it's fixed in Linus's tree but this isn't the same version of the fix that was shipped by distros.

http://seclists.org/oss-sec/2017/q2/565

With this in mind, can we expect a revised fix in the near future?

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
29 June 2017 8:04 AM

More reports confirming the Oracle DB Clusterware issue have turned up on the Oracle Community Forums

https://community.oracle.com/thread/4058097

Also OMS and OEM client

https://community.oracle.com/thread/4057668

It appears that all the affected Oracle apps modify the ulimit stack settings in startup/execution as shown in commonenv file in the OMS/OEM link above.

TM Active Contributor 113 points
29 June 2017 1:47 PM

This main article really should link to the Java/tomcat issue that PixelDrift and others have mentioned. We were just burned by the Java problem too and it took quite a bit of searching to turn up 3091371.

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
29 June 2017 10:13 PM

There is another example here:

https://access.redhat.com/solutions/3098341

Messages like the following (or core dumps) are the common symptom:

/usr/bin/uname: Argument list too long

We have found that it is almost always a 'wrapper' or equivalent 'environment' bash script that ships with the product that is modifying the ulimit setting for stack to an arbitrary number.

Something like this:

$ grep ulimit commonenv
    ulimit -S -s $EM_THREAD_STACK_SIZE
        ulimit -S -c hard

The real problem is that these static ulimit settings are all through various wrapper scripts for different products.

The above Red Hat Knowledgebase links to a Bugzilla

https://bugzilla.redhat.com/show_bug.cgi?id=1463241

Which suggests a fix exists and is currently in testing.

I think this page needs the status changed from 'resolved' with the current outstanding (known) issues.

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
6 July 2017 12:28 AM

Acknowledgement (of sorts) of the issue with the released 3.10.0-514.21.2 kernel and OracleDB 12c from Oracle here:

Stack Clash vs. Oracle Database: On the importance of testing Linux OS bug fixes and updates

pixdrift ‏‏‎ ‎'s picture Guru 6710 points
7 July 2017 1:39 AM

There is now an updated/revised fix available.. perhaps this page should be updated?

3.10.0-514.26.2.el7 changelog

2017-06-30 Frantisek Hrbata <fhrbata@hrbata.com> [3.10.0-514.26.2.el7]

    - [mm] fix new crash in unmapped_area_topdown() (Frantisek Hrbata) [1466138 1463241]
    - [mm] larger stack guard gap, between vmas (Frantisek Hrbata) [1466138 1463241]
    - [mm] Revert "enlarge stack guard gap" (Frantisek Hrbata) [1466138 1463241]
JA Newbie 4 points
26 June 2017 7:19 AM

Hi, Is this vulnerability locally exploitable only? Like I can rely on the internal controls we have in place

Reason behind is some of our environments' kernel is impacted with the vulnerability. I'm just assessing if there is a value of applying the patch now when I will be migrating them to a new environment in the coming month.

Cliff Perry's picture Red Hat Community Member 25 points
26 June 2017 9:53 AM

Hi Jay, Within the article under section - What are the attack vectors? - we state: We expect the most common form of attack will be attackers with a local account, able to control the conditions in which processes are started. Network-facing processes may be affected, and Red Hat strongly recommends upgrading the kernel and glibc to minimise these attack vectors.

While we are aware of Proof of Concepts created by Qualys that are local only attacks, we cannot exclude remote, even if that has not been proven yet.