[RHEV] My RHEV-H hypervisor suddenly lost connectivity. I see a new breth0 interface is now the default route for the host, how can I recover?
Environment
- Red Hat Enterprise Virtualization 2.2
- Both Red Hat Entperise Linux and Red Hat Enterprise Hypervisor hosts.
Issue
- My RHEV-H suddenly lost connectivity. I can't ping my host from another server in the same network.
- I see a new interface 'breth0' that has the same IP as 'rhevm' and networking isn't working anymore.
Resolution
A ful fix for this issue is currently being worked on within the following bugzilla :
Bug 754331- breth0 co-exist with rhevm bridge
https://bugzilla.redhat.com/show_bug.cgi?id=754331
The following workaround can be applied to each host where the breth0 device is present after installation :
- Place the host into maintenance mode via the RHEV-M GUI.
-
Edit the following file to disable the breth0 device at boot :
ONBOOT=no -
If this is a RHEV-H host then persist the file with the following command :
# persist /etc/sysconfig/network-scripts/ifcfg-breth0 -
Restart the networking service on the host and confirm that the breth0 device is not active and that the default route is via the rhevm device :
# service network restart # ifconfig # route
Root Cause
The following bugzilla details the root cause of this issue :
Bug 754331- breth0 co-exist with rhevm bridge
https://bugzilla.redhat.com/show_bug.cgi?id=754331
In short, the inteface should be deleted after the system starts, but due to a bug in the ovirt-node package, vdsm renames the interface on reboot and cleanup fails.
Diagnostic Steps
-
Check the following commands:
# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.10.0.0 0.0.0.0 255.255.255.0 U 0 0 0 breth0 10.10.0.0 0.0.0.0 255.255.255.0 U 0 0 0 rhevm 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 rhevm 0.0.0.0 10.10.0.254 0.0.0.0 UG 0 0 0 breth0 # ifconfig breth0 Link encap:Ethernet HWaddr 00:00:00:00:00:00 inet addr:10.10.0.31 Bcast:10.10.0.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:13083 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:549486 (536.6 KiB) ... rhevm Link encap:Ethernet HWaddr 5C:F3:FC:79:BE:FC inet addr:10.10.0.31 Bcast:10.10.0.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:48465 errors:0 dropped:0 overruns:0 frame:0 TX packets:58 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2722079 (2.5 MiB) TX bytes:4066 (3.9 KiB)
Notice that both interfaces have the same IP and route. This is wrong and should be corrected by disabling the breth0 interface.
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.
Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.
