list_add double add is detected in __list_add() called in anon_vma_chain_link() and the kernel crashes just after that.

Solution Unverified - Updated -

Issue

  • list_add double add is detected in __list_add() called in anon_vma_chain_link() and the kernel crashes just after that.
[180333.853536] ------------[ cut here ]------------
[180333.853555] WARNING: CPU: 3 PID: 17271 at lib/list_debug.c:36 __list_add+0x8a/0xc0
[180333.853559] list_add double add: new=ffff93005fb84810, prev=ffff9301793a79c0, next=ffff93005fb84810.
[180333.853562] Modules linked in: ebtable_filter ebtables ip6_tables devlink iptable_filter nfs_layout_nfsv41_files rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc sb_edac ppdev iosf_mbi crc32_pclmul vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper joydev cryptd pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw nfit vmxnet3 libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
[180333.853647] CPU: 3 PID: 17271 Comm: ksmtuned Kdump: loaded Not tainted 3.10.0-1160.11.1.el7.x86_64 #1
[180333.853650] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
[180333.853653] Call Trace:
[180333.853668]  [<ffffffffaa380faa>] dump_stack+0x19/0x1b
[180333.853679]  [<ffffffffa9c9b228>] __warn+0xd8/0x100
[180333.853685]  [<ffffffffa9c9b2af>] warn_slowpath_fmt+0x5f/0x80
[180333.853690]  [<ffffffffa9fa5caa>] __list_add+0x8a/0xc0
[180333.853699]  [<ffffffffa9e005ca>] anon_vma_chain_link+0x2a/0x40
[180333.853704]  [<ffffffffa9e02998>] anon_vma_clone+0xa8/0x1c0
[180333.853709]  [<ffffffffa9e02ae2>] anon_vma_fork+0x32/0x120
[180333.853715]  [<ffffffffa9c982c3>] dup_mm+0x453/0x760
[180333.853721]  [<ffffffffa9c99a86>] copy_process+0x1486/0x1a80
[180333.853727]  [<ffffffffa9c9a231>] do_fork+0x91/0x330
[180333.853732]  [<ffffffffa9c9a556>] SyS_clone+0x16/0x20
[180333.853741]  [<ffffffffaa394374>] stub_clone+0x44/0x70
[180333.853747]  [<ffffffffaa393f92>] ? system_call_fastpath+0x25/0x2a
[180333.853751] ---[ end trace f34fb642dfe3675a ]---
[180333.853771] BUG: unable to handle kernel paging request at 00007fab7ac59000
[180333.853864] IP: [<ffffffffa9e005b2>] anon_vma_chain_link+0x12/0x40
[180333.853919] PGD 800000031bcb5067 PUD 2e438f067 PMD 32cf43067 PTE 80000004282ff865
[180333.853986] Oops: 0003 [#1] SMP 
[180333.854033] Modules linked in: ebtable_filter ebtables ip6_tables devlink iptable_filter nfs_layout_nfsv41_files rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc sb_edac ppdev iosf_mbi crc32_pclmul vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper joydev cryptd pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw nfit vmxnet3 libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
[180333.854661] CPU: 3 PID: 17271 Comm: ksmtuned Kdump: loaded Tainted: G        W      ------------   3.10.0-1160.11.1.el7.x86_64 #1
[180333.854745] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
[180333.854822] task: ffff93032cfeb180 ti: ffff9302b2794000 task.ti: ffff9302b2794000
[180333.854877] RIP: 0010:[<ffffffffa9e005b2>]  [<ffffffffa9e005b2>] anon_vma_chain_link+0x12/0x40
[180333.854946] RSP: 0018:ffff9302b2797d00  EFLAGS: 00010246
[180333.854988] RAX: ffff93032cfeb180 RBX: 00007fab7ac59000 RCX: ffff9302b2797fd8
[180333.855041] RDX: ffff9300364ea460 RSI: 00007fab7ac59000 RDI: ffff9301793a7d80
[180333.855094] RBP: ffff9302b2797d10 R08: 000000000001f0a0 R09: ffffffffa9e02954
[180333.855148] R10: 0000000000000128 R11: 0000000000000005 R12: ffff9300364ea460
[180333.855201] R13: ffff9300364ea460 R14: ffff9301793a7d80 R15: 00007fab7ac59000
[180333.855256] FS:  00007fab81bae740(0000) GS:ffff93043fd80000(0000) knlGS:0000000000000000
[180333.855316] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[180333.855360] CR2: 00007fab7ac59000 CR3: 00000002d6ef4000 CR4: 00000000001607e0
[180333.855487] Call Trace:
[180333.855516]  [<ffffffffa9e02998>] anon_vma_clone+0xa8/0x1c0
[180333.855562]  [<ffffffffa9e02ae2>] anon_vma_fork+0x32/0x120
[180333.855608]  [<ffffffffa9c982c3>] dup_mm+0x453/0x760
[180333.855651]  [<ffffffffa9c99a86>] copy_process+0x1486/0x1a80
[180333.855697]  [<ffffffffa9c9a231>] do_fork+0x91/0x330
[180333.855740]  [<ffffffffa9c9a556>] SyS_clone+0x16/0x20
[180333.855783]  [<ffffffffaa394374>] stub_clone+0x44/0x70
[180333.855827]  [<ffffffffaa393f92>] ? system_call_fastpath+0x25/0x2a
[180333.855874] Code: 00 48 89 40 40 48 89 00 5d c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 41 54 49 89 d4 53 48 89 f3 <48> 89 3e 48 89 53 08 48 8b 57 78 48 8d 77 78 48 8d 7b 10 e8 56 
[180333.856153] RIP  [<ffffffffa9e005b2>] anon_vma_chain_link+0x12/0x40
[180333.856206]  RSP <ffff9302b2797d00>
[180333.856235] CR2: 00007fab7ac59000
  • We know another smilar crash that doesn't follow list_add double add.
[587975.208472] BUG: unable to handle kernel paging request at 000000000066d000
[587975.208646] IP: [<ffffffff961fe2e2>] anon_vma_chain_link+0x12/0x40
[587975.208730] PGD 800000017c6ed067 PUD 233e08067 PMD 233e09067 PTE 80000000b45a3865
[587975.208801] Oops: 0003 [#1] SMP 
[587975.208840] Modules linked in: ip6table_filter ip6_tables vsock_diag unix_diag af_packet_diag netlink_diag udp_diag tcp_diag inet_diag nfs_layout_nfsv41_files rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ipt_REJECT nf_reject_ipv4 vmw_vsock_vmci_transport vsock iptable_filter sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ttm lrw gf128mul glue_helper ablk_helper cryptd ppdev drm_kms_helper vmw_balloon syscopyarea sysfillrect sysimgblt fb_sys_fops drm joydev pcspkr sg drm_panel_orientation_quirks vmw_vmci i2c_piix4 parport_pc parport binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw mptspi scsi_transport_spi ata_piix mptscsih vmxnet3 libata
[587975.210110]  mptbase dm_mirror dm_region_hash dm_log dm_mod
[587975.210238] CPU: 0 PID: 6158 Comm: zabbix_agentd Kdump: loaded Not tainted 3.10.0-1062.18.1.el7.x86_64 #1
[587975.210453] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
[587975.210675] task: ffff8fa7b33aa0e0 ti: ffff8fa7b3a8c000 task.ti: ffff8fa7b3a8c000
[587975.210877] RIP: 0010:[<ffffffff961fe2e2>]  [<ffffffff961fe2e2>] anon_vma_chain_link+0x12/0x40
[587975.211087] RSP: 0018:ffff8fa7b3a8fd00  EFLAGS: 00010246
[587975.211198] RAX: 000000000066d000 RBX: 000000000066d000 RCX: 0000000041ddd175
[587975.211400] RDX: ffff8fa5b6158500 RSI: 000000000066d000 RDI: ffff8fa58da13290
[587975.211592] RBP: ffff8fa7b3a8fd10 R08: 000000000001f0a0 R09: ffffffff96200684
[587975.211786] R10: ffff8fa58da12e58 R11: 0000000000000000 R12: ffff8fa5b6158500
[587975.211983] R13: ffff8fa7b4712370 R14: ffff8fa58da13290 R15: 000000000066d000
[587975.212177] FS:  00007f5240197d80(0000) GS:ffff8fa7b5600000(0000) knlGS:0000000000000000
[587975.212377] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[587975.212489] CR2: 000000000066d000 CR3: 0000000234714000 CR4: 00000000000607f0
[587975.212709] Call Trace:
[587975.212806]  [<ffffffff962006c8>] anon_vma_clone+0xa8/0x1c0
[587975.212931]  [<ffffffff96200812>] anon_vma_fork+0x32/0x120
[587975.213058]  [<ffffffff96098be3>] dup_mm+0x453/0x730
[587975.213167]  [<ffffffff9609a34e>] copy_process+0x145e/0x1a50
[587975.213281]  [<ffffffff9609aaf1>] do_fork+0x91/0x330
[587975.213402]  [<ffffffff9678de15>] ? system_call_after_swapgs+0xa2/0x146
[587975.213523]  [<ffffffff9678de21>] ? system_call_after_swapgs+0xae/0x146
[587975.213644]  [<ffffffff9678de15>] ? system_call_after_swapgs+0xa2/0x146
[587975.213764]  [<ffffffff9678de21>] ? system_call_after_swapgs+0xae/0x146
[587975.213885]  [<ffffffff9609ae16>] SyS_clone+0x16/0x20
[587975.213999]  [<ffffffff9678e2b4>] stub_clone+0x44/0x70
[587975.214109]  [<ffffffff9678dede>] ? system_call_fastpath+0x25/0x2a
[587975.214231] Code: 00 48 89 40 40 48 89 00 5d c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 54 49 89 d4 53 48 89 f3 <48> 89 3e 48 89 53 08 48 8b 57 78 48 8d 77 78 48 8d 7b 10 e8 66 
[587975.214658] RIP  [<ffffffff961fe2e2>] anon_vma_chain_link+0x12/0x40
[587975.214778]  RSP <ffff8fa7b3a8fd00>
[587975.214874] CR2: 000000000066d000

Environment

  • Red Hat Enterprise Linux 7.9 (kernel-3.10.0-1160.11.1.el7)
  • Red Hat Enterprise Linux 7.7 (kernel-3.10.0-1062.18.1.el7)

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content