NetworkManager and firewalld - Zone is lost on network restart
Hello,
I have a firewalld config with two interfaces, each assigned to its own zone. One is internal, one is external. This has worked fine for about 6 months, but after the latest patch cycle we have an issue. I have read a couple recent threads/bugs on related topics:
- https://bugzilla.redhat.com/show_bug.cgi?id=1112742
- https://access.redhat.com/discussions/1455033
Here is the issue - replicated with test-cases. First, here is my expected config (before the patch cycle):
# firewall-cmd --get-active-zones
my_external_zone
interfaces: em1
my_internal_zone
interfaces: p1p1
Here is the issue. After the last patch cycle and reboot I have the following:
# firewall-cmd --get-active-zones
my_external_zone
interfaces: em1
drop
interfaces: p1p1
In other words, interface p1p1 lost it's zone (which was configured via nmcli and present in the network-scripts/ifcfg-p1p1 file)
Here is the test case I just ran, which makes me think there is a bug, or perhaps that I am doing something obvious wrong and not seeing my error:
First, I removed the p1p1 interface from the default drop zone:
# firewall-cmd --permanent --zone=drop --remove-interface=p1p1
The interface is under control of NetworkManager and already bound to the default zone
The interface is under control of NetworkManager, setting zone to default.
success
Then, I added the p1p1 interface to the desired zone:
# firewall-cmd --permanent --zone=my_internal_zone --add-interface=p1p1
The interface is under control of NetworkManager, setting zone to 'my_internal_zone'.
success
# firewall-cmd --reload
# firewall-cmd --get-active-zones
my_external_zone
interfaces: em1
my_internal_zone
interfaces: p1p1
Then, I verified by checking the network-scripts/ifcfg-p1p1 file:
# cat /etc/sysconfig/network-scripts/ifcfg-p1p1
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=p1p1
DEVICE=p1p1
ONBOOT=yes
IPADDR=192.168.1.1
PREFIX=24
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
ZONE=my_internal_zone
Looks good! I thought the problem was solved at this point. Nope - I restarted networking and checked again:
# systemctl restart network
# firewall-cmd --get-active-zones
my_external_zone
interfaces: em1
drop
interfaces: p1p1
# cat /etc/sysconfig/network-scripts/ifcfg-p1p1
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=p1p1
DEVICE=p1p1
ONBOOT=yes
IPADDR=192.168.1.1
PREFIX=24
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
The interface has lost its zone. I reproduced the same results with a full reboot. Then, I tried the same test using nmcli as opposed to firewall-cmd:
# nmcli connection modify p1p1 connection.zone my_internal_zone
# cat /etc/sysconfig/network-scripts/ifcfg-p1p1
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=p1p1
DEVICE=p1p1
ONBOOT=yes
IPADDR=192.168.1.1
PREFIX=24
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
ZONE=my_internal_zone
So far so good. Then, I restarted network services and checked again:
# cat /etc/sysconfig/network-scripts/ifcfg-p1p1
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=p1p1
DEVICE=p1p1
ONBOOT=yes
IPADDR=192.168.1.1
PREFIX=24
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
So, again, the interface has lost its zone after it was set via nmcli and added to the network-scripts/ifcfg-p1p1 file.
Does anyone see anything obvious I am doing wrong here? If not, this seems like a pretty significant bug that was introduced in a recent patch. The last thing in the world any of us want is for all these MANY different and inter-operating configuration commands to affect things like our network configuration, which very seldom changes and in most cases (on the server at least) really needs to stay stable and static.
Thanks for any advise you can offer - especially those of you from RedHat monitoring these discussion threads.
Responses
I had similar problems recently after the latest patch, one workaround would be to put NM_CONTROLLED=no in your ifcfg so that NetworkManager will not remove the zone anymore (if you can afford your interface not to be controlled by NM).
I confirmed this behavior on CentOS 7.2.1511 and CentOS 7.3.1611.
No matter what combination of NetworkManager or firewalld commands that I used to keep the zones persistent, nothing seemed to work. Here's the workaround that I came up with for my use case.
I have the following interfaces that should be part of the corresponding zones persistently:
enp0s3 - public
enp0s8 - internal
enp0s9 - external
I created a systemd unit file as a temporary fix to my troubles:
/etc/systemd/system/myzones.service
[Unit]
Description=Custom Zones
After=network.target network.service
[Service]
Type=oneshot
User=root
Group=root
ExecStart=/bin/nmcli connection modify enp0s3 connection.zone public
ExecStart=/bin/nmcli connection modify enp0s8 connection.zone internal
ExecStart=/bin/nmcli connection modify enp0s9 connection.zone external
[Install]
WantedBy=multi-user.target
Then I enabled the service and rebooted
# systemctl daemon-reload
# systemctl enable myzones.service
# reboot
Only issue I ran into so far is if you try and restart the network.service when there are ifcfg-*.old files residing in /etc/sysconfig/network-scripts/. But a reboot will sort that out. I don't use the network service so I can live with this temporary fix for now.
Ran into this exact problem standing up some postgres servers. The fix is to use the '--permanent' flag on firewall-cmd when making the zone change.
firewall-cmd --zone=my_internal_zone --change-interface=p1p1 --permanent
I’ve found a setting in the /etc/firewalld/firewalld.conf file “FlushAllOnReload” Changing the default from yes to no seemed to fix my issue temporarily.
Update: changing /etc/firwalld/firewalld.conf to iptables instead of the nftables may be a potential fix - note iptables will depreciate soon so hopefully nftables gets sorted
Commands run to get VPN functioning correctly are as follows
sudo firewall-cmd --zone= --add-source= --permanent sudo firewall-cmd --zone= --change-interface= --permanent sudo firewall-cmd --reload
Setup for zones: Internal-vpn for ssh Public-Ethernet