How to route traffic based on connection marks using policy-based routing?

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux - All version
  • nftables
  • iptables

Issue

  • How to route traffic based on connection marks using policy-based routing?

Resolution

Here are the steps for implementing policy-based routing based on the connection mark.

Scenario:

All incoming requests from ens192 are routed through the ens224 interface using the connection mark for all server responses.

Step 1:

  • Set all incoming connections from interface ens192 and set the connection mark to 21 using iptables or nftables

iptables rules:


# iptables -t mangle -I PREROUTING 1 -i  ens192 -j CONNMARK --set-mark 21

# iptables -t mangle -A OUTPUT -d 192.168.0.0/24 -j CONNMARK --restore-mark

nftables rules:


# nft insert rule ip mangle PREROUTING iifname "ens192" counter ct mark set 21

# nft add rule ip mangle OUTPUT ip daddr 192.168.0.0/24 counter meta mark set ct mark

Step 2:

  • Create routing rules for the connection mark to route the traffic to ens224 with table 101

# ip rule add fwmark 21 lookup 101

Step 3:

  • Add routing tables for ens224 to allow routing packets from the source IP address subnet with table 101.

# ip route add 192.168.0.0/24 oif ens224 table 101

Note:

 - Make sure to set the rp_filter value to 'Loose mode' to allow packets to be forwarded to another interface; otherwise, the kernel will drop them.

Command:

# sysctl -w net.ipv4.conf.all.rp_filter=2

Diagnostic Steps

  • Check the ip rules list using the below command
# ip rule list

0:      from all lookup local
32765:  from all fwmark 0x15 lookup 101
32766:  from all lookup main
32767:  from all lookup default
  • Check the command routing tables routes below with the table number.
# ip route show table 101

192.168.0.0/24 dev ens224 scope link
  • Check connection mark details using the below command
# cat /proc/net/nf_conntrack |grep 192.168.0.103

ipv4     2 tcp      6 431996 ESTABLISHED src=172.16.100.103 dst=172.16.100.20 sport=48880 dport=80 src=192.168.0.20 dst=192.168.0.103 sport=80 dport=48880 [ASSURED] mark=21 zone=0 use=2

ipv4     2 tcp      6 6 CLOSE src=192.168.0.103 dst=192.168.0.20 sport=38520 dport=80 src=192.168.0.20 dst=192.168.0.103 sport=80 dport=38520 [ASSURED] mark=21 zone=0 use=2
  • Check packet flow using tcpdump to see if the packets are routed correctly or not.

# tcpdump -nni ens192 icmp

listening on ens192, link-type EN10MB (Ethernet), capture size 262144 bytes
14:30:46.453886 IP 192.168.0.103 > 192.168.0.20: ICMP echo request, id 20, seq 1, length 64
14:30:47.453746 IP 192.168.0.103 > 192.168.0.20: ICMP echo request, id 20, seq 2, length 64
14:30:48.484025 IP 192.168.0.103 > 192.168.0.20: ICMP echo request, id 20, seq 3, length 64

#  tcpdump -nni ens224 icmp

listening on ens224, link-type EN10MB (Ethernet), capture size 262144 bytes

14:30:46.454051 IP 192.168.0.20 > 192.168.0.103: ICMP echo reply, id 20, seq 1, length 64
14:30:47.453905 IP 192.168.0.20 > 192.168.0.103: ICMP echo reply, id 20, seq 2, length 64
14:30:48.484183 IP 192.168.0.20 > 192.168.0.103: ICMP echo reply, id 20, seq 3, length 64

Disclaimer: The IP addresses, hostnames and MAC addresses, etc provided here are for convenience only and are sourced exclusively from the Red Hat LAB environment. You acknowledge that we are not responsible or liable for any losses or expenses that may result from the use of such IP addresses for illustrative purposes in this article.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments