PANIC: "kernel BUG at net/core/skbuff.c:131!"

Latest response

Hi,

My RHES7.6 server reboots itself once every two days!

The current version is

cat /proc/version
Linux version 3.10.0-957.21.3.el7.x86_64 (mockbuild@x86-017.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) ) #1 SMP Fri Jun 14 02:54:29 EDT 2019

This is going on for sometime (nearly a year) And vmcore shows the following:

crash /usr/lib/debug/lib/modules/3.10.0-957.21.3.el7.x86_64/vmlinux /var/crash/127.0.0.1-2019-07-09-12:53:42/vmcore

crash 7.2.3-8.el7
Copyright (C) 2002-2017 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

WARNING: kernel relocated [640MB]: patching 85670 gdb minimal_symbol values

  KERNEL: /usr/lib/debug/lib/modules/3.10.0-957.21.3.el7.x86_64/vmlinux
DUMPFILE: /var/crash/127.0.0.1-2019-07-09-12:53:42/vmcore  [PARTIAL DUMP]
    CPUS: 6
    DATE: Tue Jul  9 12:53:31 2019
  UPTIME: 2 days, 00:18:31

LOAD AVERAGE: 0.00, 0.01, 0.05
TASKS: 891
NODENAME: rhes76
RELEASE: 3.10.0-957.21.3.el7.x86_64
VERSION: #1 SMP Fri Jun 14 02:54:29 EDT 2019
MACHINE: x86_64 (3600 Mhz)
MEMORY: 63.9 GB
PANIC: "kernel BUG at net/core/skbuff.c:131!"
PID: 0
COMMAND: "swapper/3"
TASK: ffff8c03a9576180 (1 of 6) [THREAD_INFO: ffff8c03a95d0000]
CPU: 3
STATE: TASK_RUNNING (PANIC)

There don't seem to be any solution I can find. I have upgraded and downgraded kernels but no luck.

In contrast I have an old rhes 5.6.4 that is running for 10 years rock solid on an old hardware. No problem of this sort.

I trust someone in the community can help me?

Regards,

Mich

Attachments

Responses

This looks like a known problem - see System Panic Reboots with 'kernel BUG at net/core/skbuff.c:131!' though it doesn't look like there is a solution yet.

Thanks Michael.

I looked at that case you are referring. My question is do we have a community of Red Hat users that see their servers rebooted few times a week? I have two of these servers and they suffer from the same reboot issue. Any work-around by any chance?

Regards,

Mich

Hello Mich,

IMHO is best to open a support case and refer to the bugzilla mentioned in the Solution article Michael referred to.

Regards,

Jan Gerrit Kootstra

Hello Jan.

Thanks for the response. I already did that a few months back and no luck.

This is very disappointing as there is clearly a show stopper bug that is affecting a sizeable community of red hat customers. But (I trust I am not unkind to state that) no serious effort has been made to resolve this bug in a timely manner.

Regards,

Mich

Hello Mich,

I opened a support case to request a Red Hatter to look into this Discussion or open the bugzilla to the public. Mentioning my status as a Red Hat Accelerator, hope it helps.

Regards,

Jan Gerrit Kootstra

Thanks Jan. Much appreciated

It may take until tomorrow in the European afternoon (CEST) before I get a response due to a standard subscription. As soon as I get a response, I will update you.

Hope a Red Hatter will update this discussion before I do.

Hi Mich,

The support engineer states: "Red Hat is working on the bugzilla, but due to support of third parties they cannot make the bugzilla public".

The team working on it would like to get more vmcore files of crashed systems.

Could you open a support case and provide a vmcore and sosreport of a system after a crash?

Regards,

Jan Gerrit

Thanks Jan.

I will open another support case. However, I am pretty sure nothing is going to happen.

Cheers,

Mich

Successfully created case number 02423671

I closed my case, if you do not hear anything from Red Hat support within your SLA time I will reopen the case.