list_add double add is detected in __list_add() called in anon_vma_chain_link() and the kernel crashes just after that.
Issue
- list_add double add is detected in __list_add() called in anon_vma_chain_link() and the kernel crashes just after that.
[180333.853536] ------------[ cut here ]------------
[180333.853555] WARNING: CPU: 3 PID: 17271 at lib/list_debug.c:36 __list_add+0x8a/0xc0
[180333.853559] list_add double add: new=ffff93005fb84810, prev=ffff9301793a79c0, next=ffff93005fb84810.
[180333.853562] Modules linked in: ebtable_filter ebtables ip6_tables devlink iptable_filter nfs_layout_nfsv41_files rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc sb_edac ppdev iosf_mbi crc32_pclmul vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper joydev cryptd pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw nfit vmxnet3 libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
[180333.853647] CPU: 3 PID: 17271 Comm: ksmtuned Kdump: loaded Not tainted 3.10.0-1160.11.1.el7.x86_64 #1
[180333.853650] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
[180333.853653] Call Trace:
[180333.853668] [<ffffffffaa380faa>] dump_stack+0x19/0x1b
[180333.853679] [<ffffffffa9c9b228>] __warn+0xd8/0x100
[180333.853685] [<ffffffffa9c9b2af>] warn_slowpath_fmt+0x5f/0x80
[180333.853690] [<ffffffffa9fa5caa>] __list_add+0x8a/0xc0
[180333.853699] [<ffffffffa9e005ca>] anon_vma_chain_link+0x2a/0x40
[180333.853704] [<ffffffffa9e02998>] anon_vma_clone+0xa8/0x1c0
[180333.853709] [<ffffffffa9e02ae2>] anon_vma_fork+0x32/0x120
[180333.853715] [<ffffffffa9c982c3>] dup_mm+0x453/0x760
[180333.853721] [<ffffffffa9c99a86>] copy_process+0x1486/0x1a80
[180333.853727] [<ffffffffa9c9a231>] do_fork+0x91/0x330
[180333.853732] [<ffffffffa9c9a556>] SyS_clone+0x16/0x20
[180333.853741] [<ffffffffaa394374>] stub_clone+0x44/0x70
[180333.853747] [<ffffffffaa393f92>] ? system_call_fastpath+0x25/0x2a
[180333.853751] ---[ end trace f34fb642dfe3675a ]---
[180333.853771] BUG: unable to handle kernel paging request at 00007fab7ac59000
[180333.853864] IP: [<ffffffffa9e005b2>] anon_vma_chain_link+0x12/0x40
[180333.853919] PGD 800000031bcb5067 PUD 2e438f067 PMD 32cf43067 PTE 80000004282ff865
[180333.853986] Oops: 0003 [#1] SMP
[180333.854033] Modules linked in: ebtable_filter ebtables ip6_tables devlink iptable_filter nfs_layout_nfsv41_files rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc sb_edac ppdev iosf_mbi crc32_pclmul vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper joydev cryptd pcspkr sg vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx sd_mod crc_t10dif crct10dif_generic drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw nfit vmxnet3 libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
[180333.854661] CPU: 3 PID: 17271 Comm: ksmtuned Kdump: loaded Tainted: G W ------------ 3.10.0-1160.11.1.el7.x86_64 #1
[180333.854745] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
[180333.854822] task: ffff93032cfeb180 ti: ffff9302b2794000 task.ti: ffff9302b2794000
[180333.854877] RIP: 0010:[<ffffffffa9e005b2>] [<ffffffffa9e005b2>] anon_vma_chain_link+0x12/0x40
[180333.854946] RSP: 0018:ffff9302b2797d00 EFLAGS: 00010246
[180333.854988] RAX: ffff93032cfeb180 RBX: 00007fab7ac59000 RCX: ffff9302b2797fd8
[180333.855041] RDX: ffff9300364ea460 RSI: 00007fab7ac59000 RDI: ffff9301793a7d80
[180333.855094] RBP: ffff9302b2797d10 R08: 000000000001f0a0 R09: ffffffffa9e02954
[180333.855148] R10: 0000000000000128 R11: 0000000000000005 R12: ffff9300364ea460
[180333.855201] R13: ffff9300364ea460 R14: ffff9301793a7d80 R15: 00007fab7ac59000
[180333.855256] FS: 00007fab81bae740(0000) GS:ffff93043fd80000(0000) knlGS:0000000000000000
[180333.855316] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[180333.855360] CR2: 00007fab7ac59000 CR3: 00000002d6ef4000 CR4: 00000000001607e0
[180333.855487] Call Trace:
[180333.855516] [<ffffffffa9e02998>] anon_vma_clone+0xa8/0x1c0
[180333.855562] [<ffffffffa9e02ae2>] anon_vma_fork+0x32/0x120
[180333.855608] [<ffffffffa9c982c3>] dup_mm+0x453/0x760
[180333.855651] [<ffffffffa9c99a86>] copy_process+0x1486/0x1a80
[180333.855697] [<ffffffffa9c9a231>] do_fork+0x91/0x330
[180333.855740] [<ffffffffa9c9a556>] SyS_clone+0x16/0x20
[180333.855783] [<ffffffffaa394374>] stub_clone+0x44/0x70
[180333.855827] [<ffffffffaa393f92>] ? system_call_fastpath+0x25/0x2a
[180333.855874] Code: 00 48 89 40 40 48 89 00 5d c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 e5 41 54 49 89 d4 53 48 89 f3 <48> 89 3e 48 89 53 08 48 8b 57 78 48 8d 77 78 48 8d 7b 10 e8 56
[180333.856153] RIP [<ffffffffa9e005b2>] anon_vma_chain_link+0x12/0x40
[180333.856206] RSP <ffff9302b2797d00>
[180333.856235] CR2: 00007fab7ac59000
- We know another smilar crash that doesn't follow list_add double add.
[587975.208472] BUG: unable to handle kernel paging request at 000000000066d000
[587975.208646] IP: [<ffffffff961fe2e2>] anon_vma_chain_link+0x12/0x40
[587975.208730] PGD 800000017c6ed067 PUD 233e08067 PMD 233e09067 PTE 80000000b45a3865
[587975.208801] Oops: 0003 [#1] SMP
[587975.208840] Modules linked in: ip6table_filter ip6_tables vsock_diag unix_diag af_packet_diag netlink_diag udp_diag tcp_diag inet_diag nfs_layout_nfsv41_files rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ipt_REJECT nf_reject_ipv4 vmw_vsock_vmci_transport vsock iptable_filter sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ttm lrw gf128mul glue_helper ablk_helper cryptd ppdev drm_kms_helper vmw_balloon syscopyarea sysfillrect sysimgblt fb_sys_fops drm joydev pcspkr sg drm_panel_orientation_quirks vmw_vmci i2c_piix4 parport_pc parport binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel serio_raw mptspi scsi_transport_spi ata_piix mptscsih vmxnet3 libata
[587975.210110] mptbase dm_mirror dm_region_hash dm_log dm_mod
[587975.210238] CPU: 0 PID: 6158 Comm: zabbix_agentd Kdump: loaded Not tainted 3.10.0-1062.18.1.el7.x86_64 #1
[587975.210453] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
[587975.210675] task: ffff8fa7b33aa0e0 ti: ffff8fa7b3a8c000 task.ti: ffff8fa7b3a8c000
[587975.210877] RIP: 0010:[<ffffffff961fe2e2>] [<ffffffff961fe2e2>] anon_vma_chain_link+0x12/0x40
[587975.211087] RSP: 0018:ffff8fa7b3a8fd00 EFLAGS: 00010246
[587975.211198] RAX: 000000000066d000 RBX: 000000000066d000 RCX: 0000000041ddd175
[587975.211400] RDX: ffff8fa5b6158500 RSI: 000000000066d000 RDI: ffff8fa58da13290
[587975.211592] RBP: ffff8fa7b3a8fd10 R08: 000000000001f0a0 R09: ffffffff96200684
[587975.211786] R10: ffff8fa58da12e58 R11: 0000000000000000 R12: ffff8fa5b6158500
[587975.211983] R13: ffff8fa7b4712370 R14: ffff8fa58da13290 R15: 000000000066d000
[587975.212177] FS: 00007f5240197d80(0000) GS:ffff8fa7b5600000(0000) knlGS:0000000000000000
[587975.212377] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[587975.212489] CR2: 000000000066d000 CR3: 0000000234714000 CR4: 00000000000607f0
[587975.212709] Call Trace:
[587975.212806] [<ffffffff962006c8>] anon_vma_clone+0xa8/0x1c0
[587975.212931] [<ffffffff96200812>] anon_vma_fork+0x32/0x120
[587975.213058] [<ffffffff96098be3>] dup_mm+0x453/0x730
[587975.213167] [<ffffffff9609a34e>] copy_process+0x145e/0x1a50
[587975.213281] [<ffffffff9609aaf1>] do_fork+0x91/0x330
[587975.213402] [<ffffffff9678de15>] ? system_call_after_swapgs+0xa2/0x146
[587975.213523] [<ffffffff9678de21>] ? system_call_after_swapgs+0xae/0x146
[587975.213644] [<ffffffff9678de15>] ? system_call_after_swapgs+0xa2/0x146
[587975.213764] [<ffffffff9678de21>] ? system_call_after_swapgs+0xae/0x146
[587975.213885] [<ffffffff9609ae16>] SyS_clone+0x16/0x20
[587975.213999] [<ffffffff9678e2b4>] stub_clone+0x44/0x70
[587975.214109] [<ffffffff9678dede>] ? system_call_fastpath+0x25/0x2a
[587975.214231] Code: 00 48 89 40 40 48 89 00 5d c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 e5 41 54 49 89 d4 53 48 89 f3 <48> 89 3e 48 89 53 08 48 8b 57 78 48 8d 77 78 48 8d 7b 10 e8 66
[587975.214658] RIP [<ffffffff961fe2e2>] anon_vma_chain_link+0x12/0x40
[587975.214778] RSP <ffff8fa7b3a8fd00>
[587975.214874] CR2: 000000000066d000
Environment
- Red Hat Enterprise Linux 7.9 (kernel-3.10.0-1160.11.1.el7)
- Red Hat Enterprise Linux 7.7 (kernel-3.10.0-1062.18.1.el7)
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.