s390x RHEL guest crashes just after a list_del corruption in pgtable_trans_huge_withdraw() due to a possible slab use-after-free bug.

Solution Unverified - Updated -

Issue

  • s390x RHEL guest crashes just after a list_del corruption in pgtable_trans_huge_withdraw().
      KERNEL: /cores/retrace/repos/kernel/s390x/usr/lib/debug/lib/modules/3.10.0-1160.11.1.el7.s390x/vmlinux
    DUMPFILE: /cores/retrace/tasks/759395542/crash/vmcore  [PARTIAL DUMP]
        CPUS: 8
        DATE: Wed Mar 17 12:09:50 EDT 2021
      UPTIME: 52 days, 12:43:45
LOAD AVERAGE: 0.06, 0.11, 0.13
       TASKS: 1091
    NODENAME: lpdza550
     RELEASE: 3.10.0-1160.11.1.el7.s390x
     VERSION: #1 SMP Mon Nov 30 13:07:00 EST 2020
     MACHINE: s390x  (unknown Mhz)
      MEMORY: 9.8 GB
       PANIC: "Oops: 0038 [#1] SMP " (check log for details)

[4538625.877056] list_del corruption. next->prev should be 00000000ae077800, but was 3f4efcc61d3f0582
[4538625.909019] ------------[ cut here ]------------
[4538625.997501] WARNING: CPU: 6 PID: 33117 at lib/list_debug.c:62 __list_del_entry+0xa0/0xe0
[4538625.997525] Modules linked in: qeth_l3 xt_multiport xt_nat xt_addrtype xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_comment veth bridge softdog rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache sg ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_filter xt_conntrack nf_conntrack overlay(T) tcp_diag udp_diag inet_diag af_packet_diag netlink_diag appldata_net_sum unix_diag appldata_mem appldata_os nfnetlink dm_mirror dm_region_hash dm_log dm_mod vmur zcrypt_cex4 auth_rpcgss binfmt_misc sunrpc ip_tables xfs libcrc32c dasd_fba_mod dasd_eckd_mod dasd_mod pkey zcrypt ap sha512_s390 ghash_s390 des_s390 des_generic aes_s390 qeth_l2 qeth ccwgroup qdio prng 8021q garp stp llc mrp
[4538625.997637] CPU: 6 PID: 33117 Comm: G1 Conc#1 Kdump: loaded Tainted: G               ------------ T 3.10.0-1160.11.1.el7.s390x #1
[4538625.997642] task: 00000001f3d39780 ti: 00000001f0b04000 task.ti: 00000001f0b04000
[4538625.997651] Krnl PSW : 0704e00180000000 00000000004a62d0 (__list_del_entry+0xa0/0xe0)
[4538625.997656]            R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 EA:3
                 Krnl GPRS: 0000000000149a00 0000000000c5df04 0000000000000054 0000000000000000
[4538625.997665]            0000000000754a32 0000000000000000 0000000742e00000 0000000000000000
[4538625.997668]            0000000000000000 0000000742e00000 00000001f0b07c80 000003d1044e4000
[4538625.997671]            0000000100000001 00000000ae077800 00000000004a62cc 00000001f0b07980
[4538626.012559] Krnl Code: 00000000004a62c0: c02000241195  larl    %r2,9285ea
                       00000000004a62c6: c0e50015737d   brasl   %r14,7549c0
                      #00000000004a62cc: a7f40001       brc 15,4a62ce
                      >00000000004a62d0: a7f4ffdf       brc 15,4a628e
                       00000000004a62d4: b9040032       lgr %r3,%r2
                       00000000004a62d8: e34010000004   lg  %r4,0(%r1)
                       00000000004a62de: c02000241169   larl    %r2,9285b0
                       00000000004a62e4: c0e50015736e   brasl   %r14,7549c0
[4538626.012654] Call Trace:
[4538626.012656] ([<00000000004a62cc>] __list_del_entry+0x9c/0xe0)
[4538626.012659]  [<00000000004a6338>] list_del+0x28/0x40
[4538626.012666]  [<000000000012d6a2>] pgtable_trans_huge_withdraw+0x62/0xa8
[4538626.021989]  [<00000000002f24ca>] zap_huge_pmd+0x172/0x288
[4538626.022000]  [<00000000002b4db8>] unmap_page_range+0x768/0x900
[4538626.022004]  [<00000000002b4ffe>] unmap_single_vma+0xae/0x110
[4538626.022008]  [<00000000002b610a>] unmap_vmas+0x6a/0x90
[4538626.022011]  [<00000000002c054a>] exit_mmap+0xd2/0x158
[4538626.022015]  [<00000000001431d8>] mmput+0xa0/0x160
[4538626.022020]  [<000000000014eb48>] do_exit+0x2c0/0xa38
[4538626.022024]  [<000000000014f3be>] do_group_exit+0x66/0xf8
[4538626.022029]  [<0000000000164656>] get_signal_to_deliver+0x1a6/0x620
[4538626.022036]  [<000000000010adc2>] do_signal+0x92/0x7e0
[4538626.022042]  [<0000000000763ac0>] sysc_sigpending+0xa/0x12
[4538626.052488]  [<000003ff9808e854>] 0x3ff9808e854
[4538626.065917] Last Breaking-Event-Address:
[4538626.065918]  [<00000000004a62cc>] __list_del_entry+0x9c/0xe0
[4538626.065922] ---[ end trace acd4feb795258dcd ]---
[4538626.065950] Unable to handle kernel pointer dereference at virtual kernel address 3f4efcc61d3f0000
[4538626.065979] Oops: 0038 [#1] SMP 
[4538626.065983] Modules linked in: qeth_l3 xt_multiport xt_nat xt_addrtype xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_comment veth bridge softdog rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace fscache sg ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_filter xt_conntrack nf_conntrack overlay(T) tcp_diag udp_diag inet_diag af_packet_diag netlink_diag appldata_net_sum unix_diag appldata_mem appldata_os nfnetlink dm_mirror dm_region_hash dm_log dm_mod vmur zcrypt_cex4 auth_rpcgss binfmt_misc sunrpc ip_tables xfs libcrc32c dasd_fba_mod dasd_eckd_mod dasd_mod pkey zcrypt ap sha512_s390 ghash_s390 des_s390 des_generic aes_s390 qeth_l2 qeth ccwgroup qdio prng 8021q garp stp llc mrp
[4538626.066067] CPU: 6 PID: 33117 Comm: G1 Conc#1 Kdump: loaded Tainted: G        W      ------------ T 3.10.0-1160.11.1.el7.s390x #1
[4538626.066068] task: 00000001f3d39780 ti: 00000001f0b04000 task.ti: 00000001f0b04000
[4538626.066070] Krnl PSW : 0704e00180000000 00000000004a626e (__list_del_entry+0x3e/0xe0)
[4538626.066073]            R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 EA:3
                 Krnl GPRS: 000000000012d658 3f4efcc61d3f0582 000000008b7af000 0000000000000200
[4538626.066076]            abec0fafd4ef8ee5 00000000ffffffc8 0000000742f00000 0000000000000000
[4538626.066077]            0000000000000000 0000000742f00000 00000001f0b07c80 000003d1044d4000
[4538626.066079]            0000000100000001 000000008b7af000 00000001f0b07998 00000001f0b07980
[4538626.066086] Krnl Code: 00000000004a625e: ec43001e8064  cgrj    %r4,%r3,8,4a629a
                       00000000004a6264: a7390200       lghi    %r3,512
                      #00000000004a6268: ec1300458064   cgrj    %r1,%r3,8,4a62f2
                      >00000000004a626e: e32010000020   cg  %r2,0(%r1)
                       00000000004a6274: a7740030       brc 7,4a62d4
                       00000000004a6278: e32040080020   cg  %r2,8(%r4)
                       00000000004a627e: a774001c       brc 7,4a62b6
                       00000000004a6282: e31040080024   stg %r1,8(%r4)
[4538626.066101] Call Trace:
[4538626.066103] ([<00000001f0b07c80>] 0x1f0b07c80)
[4538626.066105]  [<00000000004a6338>] list_del+0x28/0x40
[4538626.066107]  [<000000000012d6a2>] pgtable_trans_huge_withdraw+0x62/0xa8
[4538626.066109]  [<00000000002f24ca>] zap_huge_pmd+0x172/0x288
[4538626.066111]  [<00000000002b4db8>] unmap_page_range+0x768/0x900
[4538626.066113]  [<00000000002b4ffe>] unmap_single_vma+0xae/0x110
[4538626.066114]  [<00000000002b610a>] unmap_vmas+0x6a/0x90
[4538626.066116]  [<00000000002c054a>] exit_mmap+0xd2/0x158
[4538626.066118]  [<00000000001431d8>] mmput+0xa0/0x160
[4538626.066120]  [<000000000014eb48>] do_exit+0x2c0/0xa38
[4538626.066122]  [<000000000014f3be>] do_group_exit+0x66/0xf8
[4538626.066124]  [<0000000000164656>] get_signal_to_deliver+0x1a6/0x620
[4538626.066126]  [<000000000010adc2>] do_signal+0x92/0x7e0
[4538626.066128]  [<0000000000763ac0>] sysc_sigpending+0xa/0x12
[4538626.066130]  [<000003ff9808e854>] 0x3ff9808e854
[4538626.066132] Last Breaking-Event-Address:
[4538626.066133]  [<00000000004a6332>] list_del+0x22/0x40
[4538626.066135]  
[4538626.066136] Kernel panic - not syncing: Fatal exception: panic_on_oops

Environment

  • rhel7.9.z kernel-3.10.0-1160.11.1.el7.s390x
  • RHEL guest running on IBM z/VM OS

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content