Receive ARP Packet Problem

Latest response

Hi all,
We have there physical server which RHEL 7.6 is installed on the servers. The servers are HPE DL580 G10, all hardware and configurations are same.
There is a problem on two of the servers, none of application servers can Ping those servers but they can Ping third server.
When I run "arping -I bond0 x.x.x.x", the target be able to Ping bond's IP but can't receive ICMP reply from VIP addresses.
When I run "tcpdump", all addresses will reachable.
Anyone had the same issue?

Responses

When you run tcpdump everything works. That's interesting as tcpdump enables promiscuous mode, which stops filtering based on MAC address.

So I guess you have some strangeness going on with MAC addresses, possibly from the VIP moving around and having different source MACs?

I would packet capture on each system's bond at the same time, use tcpdump --no-promiscuous-mode to NOT enable promisc which should reproduce the issue. Collect working and non-working ping with a datestamp at the same time. Collect netstat -s with a datestamp at the same time (our monitor.sh script can do this)

Try to understand which traffic is/isn't replied to, check the traffic to see if MAC addresses are odd or unexpected, check netstat to perhaps see a counter change which explains it.

At the risk of stating the obvious, if you have a VIP which moves around, then you need to have something which announces where the VIP is when it moves. Cluster software usually does this by sending a Gratuitous ARP such as with arping. If you're just moving the address around with ip addr del and ip addr add that is not sufficient to relocate the address.

VIP is actually for Oracle RAC. I forgot mention that all network ports are connected to two network switch, both public and private for Oracle RAC but subnets are different for public and private networks. I'm sure that bond doesn't reply ARP request. But I don't know, what's root cause because there is another RAC with same OS, hardware and configuration which there was no same problem.

Also I don't know answer of your question, I should check it again:

"So I guess you have some strangeness going on with MAC addresses, possibly from the VIP moving around and having different source MACs?"

Seems, there is a blocker for broadcast packets, how can we detect it?