How to connect two network interfaces on the same subnet?

Is there a typo in step 2) ??

Should these lines be identical?

# ip route add 10.64.208.0/24 dev eth0 src 10.64.208.180 table t1



    # ip route add 10.64.208.0/24 dev eth0 src 10.64.208.208 table t1

or shouldn't the second 'ip route add' look like this...

# ip route add 10.64.208.0/24 dev eth1 src 10.64.208.208 table t2

2. Add routing table to t1, t2 :

# ip route add 10.64.208.0/24 dev eth0 src 10.64.208.180 table t1
# ip route add table t1 default via 10.64.208.254 dev eth0
# ip route show table t1
10.64.208.0 dev eth0  scope link  src 10.64.208.180
default via 10.64.208.254 dev eth0



    # ip route add 10.64.208.0/24 dev eth0 src 10.64.208.208 table t1
    # ip route add table t2 default via 10.64.208.254 dev eth1
    # ip route show table t2
    10.64.208.0 dev eth1  scope link  src 10.64.208.208
    default via 10.64.208.254 dev eth0

.

There is also anotehr typo...

As for 4):

# grep arp_filter /etc/sysconfig 
net.ipv4.conf.all.arp_filter = 1 

should say

# grep arp_filter /etc/sysctl.conf 
net.ipv4.conf.all.arp_filter = 1

This script automatically implements the changes and creation of static routes described above, in case anyone wants to use it.

All you need to provide is the subnet that you want to effect.

#!/bin/sh

#    fixFlux.sh - Assigns fixed routes for each NIC

#                 residing on the same subnet to resolve

#                 the ARP FlUX problem

# See https://access.redhat.com/kb/docs/DOC-59096

#

#    Date: 7/21/2011

SCRIPT_REVISION=1.0.1

SCRIPT_LOG=/dev/nul

#    Author:  Ken Banks

#    Company: Kontron America.

#

# CHANGE LOG:

# 7/21/2011    1.0.1  Original code.

# 8/23/2011    1.0.2  Fix a bug with 'undo' to re-enble NetworkManager

#                     and another bug that allowed multiple 'no's to be

#                     added to NM_CONTROLLED= if the script was run more than once.

SYSCTL=/etc/sysctl.conf

RT_TABLES=/etc/iproute2/rt_tables

function usage() {

    echo "usage:  $0 subnet <undo>"

    echo "        version $SCRIPT_REVISION"

    echo "        : This script will solve the Arp Flux for multiple NICs"

    echo "        : residing on the same subnet, as described in this document:"

    echo "        : https://access.redhat.com/kb/docs/DOC-59096"

    echo

    echo " e.g.   $0 192.168.1      -  Configures a static route for ever IC on 192.168.1.0"

    echo "        $0 192.168.1 undo -  Reverts the changes caused above and re-instates NetworkManager"

}

function log_message() {

    echo -e "$*"

    echo -e "$*"  >>  $SCRIPT_LOG

}

function  compareValues() {

    awk "BEGIN { if ( $1 $2 $3 ) print \"TRUE\" }"

}

function addRoute() {

    NICNAME=$1

    TABLENUM=$2

    TABLENAME=$3

    SUBNET=$4

    DEFAULTGW=$5

    NICIP=$6

    FirstOctect=`echo $SUBNET | awk -F. '{ print $1 }'`

    ip route show  | grep $6 > /dev/nul

    if [ "$?" != "0" ] ; then

        echo ip route add  $SUBNET dev $NICNAME  src $NICIP table $TABLENAME

        ip route add  $SUBNET dev $NICNAME  src $NICIP table $TABLENAME

        echo ip route add table $TABLENAME default via $DEFAULTGW dev $NICNAME

        ip route add table $TABLENAME default via $DEFAULTGW dev $NICNAME

        #echo ip route show table $TABLENAME

        #ip route show table $TABLENAME

    fi

    ETHROUTE=/etc/sysconfig/network-scripts/route-$NICNAME

    echo $FirstOctect.0.0.0/8 dev $NICNAME src $NICIP table  $TABLENAME > $ETHROUTE

    echo default via $DEFAULTGW dev $NICNAME table  $TABLENAME >> $ETHROUTE

    echo "Modified: $ETHROUTE"

    ETHRULE=/etc/sysconfig/network-scripts/rule-$NICNAME

    echo table $TABLENAME from $NICIP > $ETHRULE

    echo "Modified: $ETHRULE"

}

function disableNetworkManager() {

    service NetworkManager status | grep stopped >/dev/nul

    if [ "$?" != "0" ] ; then

      service NetworkManager stop   > /dev/nul

    fi

    echo "NetworkManager service is stopped."

    chkconfig NetworkManager --list | grep :on >/dev/nul

    if [ "$?" == "0" ] ; then

        chkconfig NetworkManager off  > /dev/nul

    fi

    echo "NetworkManager service is disabled."

        ETHSCRIPT=/etc/sysconfig/network-scripts/ifcfg-$1

    TMPSCRIPT=/tmp/tmpcfg-$1

    grep -i NM_CONTROLLED=[\"yes\"]  $ETHSCRIPT | awk -F. '{ print $1 }'  > /dev/nul

    if [ "$?" == "0" ] ; then

        cp $ETHSCRIPT  $TMPSCRIPT

        cat $TMPSCRIPT | sed  's/NM_CONTROLLED=["yes"].*/NM_CONTROLLED=no/' > $ETHSCRIPT

        echo "Modified: $ETHSCRIPT"

    fi

}

function processNIC() {

#     inet 192.168.6.99/24 brd 192.168.6.255 scope global eth0show table

#     inet 192.168.6.60/24 brd 192.168.6.255 scope global eth1

    export NICNAME=$1

    export TABLENUM=$2

    export TABLENAME=$3

    export SUBNET=$4

    export DEFAULTGW=`ip route | grep default | awk '{ print $3 }'`

    export NICDATA=`ifconfig -a $NICNAME | grep Bcast`

    NICIP=`echo $NICDATA | awk '{ print $2 }' | awk -F: '{ print $2 }'`

    if [ "$NICIP" == "" ] ; then

        echo "ERROR: No IP for this NIC: $NICNAME"

        exit

    fi

    echo "====PROCESS NIC==== $NICDATA"

    echo "====  $NICNAME ===="

    grep $TABLENAME $RT_TABLES > /dev/nul

    if [ "$?" != "0" ] ; then

        echo "Creating route table: $TABLENUM  $TABLENAME"

        echo "$TABLENUM  $TABLENAME"  >> $RT_TABLES

    else

        echo "Route table already exists: $TABLENUM  $TABLENAME"

    fi

    addRoute $NICNAME $TABLENUM $TABLENAME $SUBNET  $DEFAULTGW $NICIP

    disableNetworkManager $NICNAME

    echo

}

function delete_route() {

   ip route delete $1 dev $2

}

function undo_routes() {

      SUBNET=$1

      echo 0 > /proc/sys/net/ipv4/conf/default/arp_filter

      echo 0 > /proc/sys/net/ipv4/conf/all/arp_filter

      cp $RT_TABLES /tmp/rt_tables

      grep -v "^.00  t[0-9]*" /tmp/rt_tables > $RT_TABLES

      rm -f /etc/sysconfig/network-scripts/r[ou][ul][te]*

      for i in `ip route show | grep src | awk '{ print $1"-"$3 }'` ; do

      delete_route  `echo $i | sed 's/-/ /'`

      done

        grep net.ipv4.conf.all.arp_filter $SYSCTL  > /dev/nul

    if [ "$?" == "0" ] ; then   

        grep  "arp_filter[ =]*0" $SYSCTL  > /dev/nul

        if [ "$?" != "0" ] ; then   

            cp $SYSCTL  /tmp/tmpctl.conf

            cat /tmp/tmpctl.conf | sed 's/arp_filter[ =]*./arp_filter = 0/' >> $SYSCTL

            echo "Modified: $SYSCTL"

        fi

    fi

      for device in `ip addr | grep $SUBNET | awk '{ print $7 }'` ; do

        ETHSCRIPT=/etc/sysconfig/network-scripts/ifcfg-$device

    TMPSCRIPT=/tmp/tmpcfg-$device

    grep -i NM_CONTROLLED=[\"no\"]  $ETHSCRIPT | awk -F. '{ print $1 }'  > /dev/nul

    if [ "$?" == "0" ] ; then

        cp $ETHSCRIPT  $TMPSCRIPT

        cat $TMPSCRIPT | sed  's/NM_CONTROLLED=[\"no\"].*/NM_CONTROLLED=yes/' > $ETHSCRIPT

        echo "Modified: $ETHSCRIPT"

    fi

      done

      chkconfig NetworkManager on

      service NetworkManager start

}

function main() {

    [ "$1" == "" ] && usage && exit

    SUBNET=$1

    [ "$2" == "undo" ] && undo_routes $SUBNET  && exit

    echo "# $0 $*"

    log_message  "=========================================================================="

    log_message  "=============================BEGIN ROUTE CREATION========================="

    log_message  "=========================================================================="

    table=1

    for device in `ip addr | grep $SUBNET | awk '{ print $7 }'` ; do

        processNIC $device $((100 * $table))  t$table  $SUBNET

        table=$((1 + $table))

    done

    echo 1 > /proc/sys/net/ipv4/conf/default/arp_filter

    echo "set /proc/sys/net/ipv4/conf/default/arp_filter = 1"

    echo 1 > /proc/sys/net/ipv4/conf/all/arp_filter

    echo "set /proc/sys/net/ipv4/conf/all/arp_filter = 1"

        grep net.ipv4.conf.all.arp_filter $SYSCTL  > /dev/nul

    if [ "$?" != "0" ] ; then   

        echo "net.ipv4.conf.all.arp_filter = 1" >>  $SYSCTL

        echo "Modified: $SYSCTL"

    else

        grep  "arp_filter[ =]*1" $SYSCTL  > /dev/nul

        if [ "$?" != "0" ] ; then   

            cp $SYSCTL  /tmp/tmpctl.conf

            cat /tmp/tmpctl.conf | sed 's/arp_filter[ =]*./arp_filter = 1/' >> $SYSCTL

            echo "Modified: $SYSCTL"

        fi

    fi

    service network restart

    log_message  "=========================================================================="

    log_message  "==========================ROUTES HAVE BEEN CREATED========================"

    log_message  "=========================================================================="

    echo "Contents of file: $RT_TABLES"

    cat $RT_TABLES

    echo "Command output from: ip route show"

    ip route show

}

main $*

the prefix was 24 at the beginning, but became 8 in the next step,way ? and I need to run "ip route add default via 10.64.208.254 dev eth1 metric 101" to make this routes persistent.

The section with dot point "Add rules to apply traffic to the routing tables". Shows how to add an "ip rule" but does not have the correct command for displaying the newly added rule.

eg//

# ip rule add table t1 from 10.64.208.180       <- OK
# ip rule add table t2 from 10.64.208.208       <- OK

# ip route show                                                         <- Does not help.  Where is the rule just added?
10.64.208.0/24 dev eth0  proto kernel  scope link  src 10.64.208.180
10.64.208.0/24 dev eth1  proto kernel  scope link  src 10.64.208.208
169.254.0.0/16 dev eth1  scope link
default via 10.64.208.254 dev eth0

After adding a "rule", it would be better to run a "ip rule show" rather than "ip route show" eg//

# ip rule show
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

# ip rule add table em1 from 172.16.13.149

# ip rule show
0:      from all lookup local
32765:  from 172.16.13.149 lookup em1   <- Here is the "rule" added via "ip rule add"
32766:  from all lookup main
32767:  from all lookup default

Hello,

I have re-written the Resolution section substantially. As a result, many of the previous comments may no longer make sense as they applied to the steps that were in the Resolution section before the change. The new Resolution section provides links to existing content in the knowledge base that provide more implementation details and example commands.