redhat EL6.5 firewall/route - crash shortly after restarting iptables

Latest response

Hi,

We have recently built an edge firewall/route using RHEL6.5 (2.6.32-431.1.2.el6.x86_64) running on a Dell PowerEdge R610 (2x Xeon E5530, 12GB, Broadcom Quad port NIC). Network driver: bnx2: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.2.3 (June 27, 2012)

We're using iptables/ip6tables (netfilter) for our firewall, and various other services such as radvd, dhcp and squid. It's handling about 100Mb/s and about 2500 devices at its peak. Conntrack is tracking about 30000 connections (/proc/sys/net/netfilter/nf_conntrack_count).

In 3 weeks we've experienced 4 crashes that have required a cold reboot. I've attached a screen shot of one of the crashes.

The Xeons in the server have a bug in them to do with returning from deep C states - I thought this was the problem initially. I've used intel_idle.max_cstate=1 grub.conf to stop deep c states being used.

However the server has just crashed again about 15 mins after I made changes to our iptables config and restarted the service. One of the previous crashed was prompted by a restart of iptables.

Can anyone suggest any avenues of investigation to solve this? I have to admit I'm not that experienced when it comes to investigating crashes on Linux!

Many thanks,
Daniel

Responses

I've configure kdump on the server so hopefully next time it crashes I'll have a core dump to analyse!

Are there any known issues with iptables being restarted under load?

Hi Daniel,

Have you opened a Red Hat case?
Analysis a kdump crash "log" is not an easy job.

It might be a security issue too, to publish a kdump crash "log".

Kind regards,

Jan Gerrit