Why IP of a shutdown system gets alloted to unknown MAC to another system?

Solution Verified - Updated -

Environment

  • Red hat Enterprise Linux 5
  • IBM System x3650
  • Cisco 3550-12G switch

Issue

  • IP of a shutdown system get alloted to unknown MAC to another system.
  • IBM Server IBM SystemX 3650 with rhel 5.7 OS has been configured with Bonded NIC Interface in adaptive Load Balancing Mode. Once the DELL system is shutdown which have the IP the same IP get assigned to a new MAC and arp table in linux and switch arp table shows it.

Resolution

Change Baseboard Management Controller(BMC) IP of IBM System x3650 to non conflicting IP or use DHCP to avoide IP conflict.

Root Cause

IBM Server SystemX 3650 Baseboard Management Controller (BMC) was configured to have a static IP which was conflicting with IP being asigned to other server.

Diagnostic Steps

  • Collect tcpdumps
  • Check IMM of IBM machine to see if the MAC is of the IMM.
  • Write system to see if the arp reply is from the server or not to eliminate linux system.
  • To see that we have duplicate IP addresses configured we can use a tool arp-scan.
    To install arp-scan run following command as root
   $ yum install -y arp-scan

Once installed please use this tool as follows to see whether we are able to locate the other host configured with same IP address having the MAC address in question

   $ arp-scan --localnet -I <interface name>

Redirect the output of the arp-scan command to a file and attach it to the case for analysis.

A. To scan the configured subnet

   $ arp-scan --localnet -I <interface name> -W /tmp/arp_scan_capture.pcap

It'll provide output in following format if it finds duplicate IP

<IP-1>  <MAC-1>  <hostname or unknown>
<IP-1>  <MAC-2>  <hostname or unknown> (Dup 1)

This will indicate that indeed we have another host in the same subnet with same IP address.

B. To scan only the interested MAC addresses. e.g for the current configuration

   $ arp-scan --destaddr <MAC address> -W /tmp/arp-scan-capture.pcap > /tmp/arp-scan-console.txt

For other host:

   $ arp-scan --destaddr <MAC address> -W /tmp/arp_scan_capture_02.pcap >/tmp/arp_scan_console_02.txt

Attach the pcap and .txt files thus generated from steps A and B in /tmp directory to the ticket for further analysis.

  • Check whether IBM Server's Baseboard Management Console (BMC) or IIM are configured to have the conflicting IP

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments