Select Your Language

Infrastructure and Management

Cloud Computing

Storage

Runtimes

Integration and Automation

  • Comments
  • RHEL as Router is not Forwarding

    Posted on

    Summary:
    We have a RHEL 9.5 VM (hostname: NuSquid) with two NICs that is configured as a router with Squid 5.5 to provide internet access to a Nutanix cluster that's on a private subnet. NuSquid's external NIC faces the Internet and works fine to allow NuSquid complete Internet access. Despite configuring NuSquid's IP forwarding, firewall rules, and Squid, the Nutanix nodes cannot access the internet thru NuSquid. Disabling firewalld and/or Squid did not resolve the issue, indicating a potential problem with IP forwarding or routing. -Hoping a contributor will recommend a solution to our dilemma!

    Computer & Network Configurations:
    * Subnets
    internet facing: 10.208.122.0/24
    internal/private: 172.20.0.0/24
    * Nutanix Cluster:
    - Node 1: 172.20.0.160 (AHV), 172.20.0.161 (CVM)
    - Node 2: 172.20.0.162 (AHV), 172.20.0.163 (CVM)
    - Node 3: 172.20.0.164 (AHV), 172.20.0.165 (CVM)
    * Nutanix`.160
    (-this instance is of the above cluster and is used for testing Internet access/routing)
    hostname: NTNX-71...
    OS: AHV (Acropolis Hypervisor) instance
    only NIC
    - device: br0
    - IP address: 172.20.0.0.160/24
    ip route show:
    default via 172.20.0.8 dev br0
    169.254.0.0/16 dev br0 scope link metric 1019
    172.20.0.0/24 dev br0 proto kernel scope link src 172.20.0.160
    192.168.5.0/24 dev virbr0 proto kernel scope link src 192.168.5.1
    * RHEL/v9.5 as Router
    hostname: NuSquid
    OS: RHEL 9.5
    Internet-facing NIC:
    - dev: ens192
    - ip addr: 10.208.122.8
    Internal/Private NIC:
    - dev: ens256
    - ip addr: 172.20.0.8
    ~]# sysctl net.ipv4.ip_forward
    net.ipv4.ip_forward = 1
    ~]# ip route show
    default via 10.208.122.1 dev ens192 proto static metric 100
    10.208.1.0/24 via 10.208.122.1 dev ens192 proto static metric 100
    10.208.122.0/24 dev ens192 proto kernel scope link src 10.208.122.8 metric 100
    172.20.0.0/24 dev ens256 proto kernel scope link src 172.20.0.8 metric 101
    ~]# firewall-cmd --get-active-zones
    t-all
    firewall-cmd --zone=external --list-all
    firewall-cmd --zone=internal --list-all
    external
    interfaces: ens192
    internal
    interfaces: ens256
    ~]# firewall-cmd --zone=external --list-all
    external (active)
    target: default
    icmp-block-inversion: no
    interfaces: ens192
    sources:
    services: ssh
    ports:
    protocols:
    forward: yes
    masquerade: yes
    forward-ports:
    source-ports:
    icmp-blocks:
    rich rules:
    ~]# firewall-cmd --zone=internal --list-all
    internal (active)
    target: default
    icmp-block-inversion: no
    interfaces: ens256
    sources:
    services: cockpit dhcpv6-client http https mdns samba-client ssh
    ports: 3128/tcp 3129/tcp 80/tcp 443/tcp
    protocols:
    forward: yes
    masquerade: yes
    forward-ports:
    port=80:proto=tcp:toport=3128:toaddr=172.20.0.8
    port=443:proto=tcp:toport=3129:toaddr=172.20.0.8
    source-ports:
    icmp-blocks:
    rich rules:

    Squid Configuration (squid.conf):
    acl NutanixCluster src 172.20.0.160-172.20.0.165/32 # Nutanix Cluster
    acl InfraMgmt src 172.20.0.0/24 # IM600
    acl iWAN src 10.208.122.0/24 # Internet facing
    acl RA-DNS src 10.208.1.0/24 # Rockwell's DNS Servers

    acl SSL_ports port 443
    acl Safe_ports port 80 # http
    acl Safe_ports port 21 # ftp
    acl Safe_ports port 443 # https
    acl Safe_ports port 70 # gopher
    acl Safe_ports port 210 # wais
    acl Safe_ports port 1025-65535 # unregistered ports
    acl Safe_ports port 280 # http-mgmt
    acl Safe_ports port 488 # gss-http
    acl Safe_ports port 591 # filemaker
    acl Safe_ports port 777 # multiling http

    http_access deny !Safe_ports

    http_access deny CONNECT !SSL_ports

    http_access allow NutanixCluster
    http_access allow InfraMgmt
    http_access allow iWAN
    http_access allow RA-DNS
    http_access deny all

    http_port 172.20.0.8:3128
    https_port 172.20.0.8:3129 cert=/etc/squid/ssl_cert/selfsigned.crt key=/etc/squid/ssl_cert/private.key

    cache_dir ufs /var/spool/squid 500 16 256
    coredump_dir /var/spool/squid

    refresh_pattern ^ftp: 1440 20% 10080
    refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
    refresh_pattern . 0 20% 4320

    Test Results on NTNX`.160:
    ~]# nc -zv 172.20.0.8 3129
    Connection to 172.20.0.8 3129 port [tcp/netport-id] succeeded!

    ~]# nc -zv 172.20.0.8 3128
    Connection to 172.20.0.8 3128 port [tcp/squid] succeeded!

    ~]# ping 8.8.8.8
    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    From 172.20.0.8 icmp_seq=1 Packet filtered
    From 172.20.0.8 icmp_seq=2 Packet filtered
    ^C
    --- 8.8.8.8 ping statistics ---
    2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1030ms

    ~]# ping dell.com
    PING dell.com (143.166.136.12) 56(84) bytes of data.
    From _gateway (172.20.0.8) icmp_seq=1 Packet filtered
    From _gateway (172.20.0.8) icmp_seq=2 Packet filtered
    ^C
    --- dell.com ping statistics ---
    2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1000ms

    ~]# curl -I http://dell.com
    HTTP/1.1 301 Moved Permanently
    Location: https://www.dell.com/
    Server: BigIP
    Content-Length: 0
    Date: Mon, 30 Dec 2024 20:20:43 GMT
    X-Cache: MISS from NuSquid
    X-Cache-Lookup: MISS from NuSquid:3128
    Via: 1.1 NuSquid (squid/5.5)
    Connection: keep-alive

    NuSquid logs after test executions on NTNX`.160:
    ~]# journalctl -u firewalld
    Jan 01 11:59:21 NuSquid systemd[1]: Starting firewalld - dynamic firewall daemon...
    Jan 01 11:59:23 NuSquid systemd[1]: Started firewalld - dynamic firewall daemon.
    Jan 01 12:16:38 NuSquid systemd[1]: Stopping firewalld - dynamic firewall daemon...
    Jan 01 12:16:39 NuSquid systemd[1]: firewalld.service: Deactivated successfully.
    Jan 01 12:16:39 NuSquid systemd[1]: Stopped firewalld - dynamic firewall daemon.
    Jan 01 12:17:10 NuSquid systemd[1]: Starting firewalld - dynamic firewall daemon...
    Jan 01 12:17:10 NuSquid systemd[1]: Started firewalld - dynamic firewall daemon.

    ~]# tail -f /var/log/squid/access.log
    1735658002.714 77 172.20.0.160 TCP_MISS/301 260 HEAD http://dell.com/ - HIER_DIRECT/143.166.136.12 -
    1735658034.735 0 172.20.0.160 NONE_NONE/000 0 - error:transaction-end-before-headers - HIER_NONE/- -
    1735658040.185 0 172.20.0.160 NONE_NONE/000 0 - error:transaction-end-before-headers - HIER_NONE/- -
    1735658116.277 0 172.20.0.160 NONE_NONE/000 0 - error:transaction-end-before-headers - HIER_NONE/- -
    1735658120.687 0 172.20.0.160 NONE_NONE/000 0 - error:transaction-end-before-headers - HIER_NONE/- -
    1735671803.983 82 172.20.0.160 TCP_MISS/301 260 HEAD http://dell.com/ - HIER_DIRECT/143.166.30.172 -
    1735750686.047 95 172.20.0.160 TCP_MISS/301 260 HEAD http://dell.com/ - HIER_DIRECT/143.166.30.172 -
    1735751316.530 77 172.20.0.160 TCP_MISS/301 260 HEAD http://dell.com/ - HIER_DIRECT/143.166.136.12 -
    ^C

    ~]# tail -f /var/log/squid/cache.log
    2025/01/01 11:59:25 kid1| Store logging disabled
    2025/01/01 11:59:25 kid1| DNS Socket created at [::], FD 10
    2025/01/01 11:59:25 kid1| DNS Socket created at 0.0.0.0, FD 11
    2025/01/01 11:59:25 kid1| Adding nameserver 10.208.1.12 from /etc/resolv.conf
    2025/01/01 11:59:25 kid1| Adding nameserver 10.208.1.14 from /etc/resolv.conf
    2025/01/01 11:59:25 kid1| HTCP Disabled.
    2025/01/01 11:59:25 kid1| Finished loading MIME types and icons.
    2025/01/01 11:59:25 kid1| Accepting HTTP Socket connections at conn15 local=172.20.0.8:3128 remote=[::] FD 12 flags=9
    2025/01/01 11:59:25 kid1| Accepting HTTPS Socket connections at conn17 local=172.20.0.8:3129 remote=[::] FD 13 flags=9
    2025/01/01 11:59:26 kid1| storeLateRelease: released 0 objects
    ^C

    by

    points

    Responses

    Red Hat LinkedIn YouTube Facebook X, formerly Twitter

    Quick Links

    Help

    Site Info

    Related Sites

    © 2025 Red Hat, Inc.