Issue with Bonding - Device ethX has a different MAC address than expected

Latest response

I’m encountering an issue while restarting/activating the network service with bonded interfaces on a RHEL 6.6 server. Details below:

  • The bonded interface and the slaves (active and backup) come up without any issues during system reboots. However, if the network service is restarted or if one of the interface went/brought down and is brought back, the backup slave does not come up – the error is: device ethX has a different MAC address than expected…ignoring
  • If I remove the HWADDR directive from the interface configuration files of the slaves, then this issue does not occur. However, the RedHat documentation mandates that the HWADDR directive be included in the interface configuration files if more than one NIC is present in the host. (the BL660c Gen8 has 16 interfaces btw).
  • The udev rules are in place and match with the interface configuration files.
  • I’ve already gone through the steps in https://access.redhat.com/solutions/223433 except for contacting the vendor

Could you please advise how I can go about this?

Sample config files:

== Ifcfg-bondX==
DEVICE=bondX
IPADDR=x.x.x.x
NETMASK=x.x.x.x
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
NM_CONTROLLED=no
BONDING_OPTS=”mode=1 miimon=100”

== Ifcfg-ethX ==
DEVICE=ethX
BOOTPROTO=none
ONBOOT=yes
NM_CONTROLLED=no
USERCTL=no
HWADDR=x.x.x.x.x.x
UUID=x.x.x.x..x.x.x.x.x.x
MASTER=bondX
SLAVE=yes

I've reviewed this article as well: https://access.redhat.com/solutions/336513

The article requires you to replace the HWADDR suggested by ‘ip a show ethX’ output. In reality, this MAC address is from the Primary/Active Slave interface. AFAIK, the bondX interface chooses the MAC address of the Primary interface (among the bonded interfaces) and assigns/expects the same to secondary/backup slave interface to avoid packet drops during failover.

When I checked the contents of /sys/class/net/eth{1,8}/address, I see that both have the same MAC address.

My concern is:
- Can you activate successfully two interfaces with the same MAC addresses???
- Can we remove HWADDR from the slave interface configuration files? will that have any other impact?

Responses

Hello

your questions: 1] You must not put the same MAC address in two different config files. The HWADDR directive ties the config file to the device with that address. 2] Yes, you can remove the HWADDR directive. You can leave out the directive and let the system assign a config file to a device.

==>2 but my concern is that Red Hat documentation mentions that HWADDR directive should be present if the host has more than one NIC. Just wondering what impact this will have if I remove the HWADDR from the interface files

The interface config files will be applied to NICs in an unpredictable way at boot time. So you will never be sure what NIC is assigned to what EthX config at boot time. I think its best to raise a support case after first contacting the vendor for an updated driver.

Isn't that the reason the udev rules are applied?

I believe so, but I will asks colleagues what will happen without HWADDR as I have never tested that.

Colleague confirms: Unlike RHEL5, there is no need for HWADDR. Consistent Device Naming is provided in RHEL 6 when the OS udev rules generate the 70-persistent-net.rules file which permanently maps a specific ethX name onto an interface based on its MAC address.

You mentioned the contents of /sys/class/net/eth{1,8}/address being the same MAC address.

When a bond comes up, it takes its MAC address from the MAC address of the first slave and applies that to the other slaves.

EDIT: TLB and ALB mode do require a unique MAC address for each slave.

Please tell us what make and model of NICs you have.

it's a Qlogic FLB 630 20G

I remember a recent Qlogic and bonds related fix in Red Hat Enterprise Linux 7. Its in the release notes Setting up bonding with qlcnic no longer fails. I searched https://bugzilla.redhat.com but could not see anything related for Red Hat Enterprise Linux 6.

I suggest contact vendor and ask for driver that supports Red Hat Enterprise Linux 6.6 or later.

Is using Red Hat Enterprise Linux 7 an option for you?

Further info from P Talbert:

If you do include a HWADDR in an ifcfg file then two things happen. An early udev rule checks all ifcfg files for the HWADDR param and if it finds one in an ifcfg file it then checks for DEVICE. If both are found, it will always try to name the interface with the matching MAC to the name given in by the DEVICE param in the matching ifcfg file.

The other effect of having HWADDR set in an ifcfg file, which it sounds like the problem here, is that the network service and NM will now only apply this configuration if they find an interface with a matching MAC. If you also have DEVICE set in the file, then that must match as well. It makes the check on whether a configuration defined in an ifcfg file is used more stringent.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.