Enable Asymmetric Routing

Asymmetric Routing means packets from A to B follow different routes then packets from B to A.
If a machine receives packets arriving on one interface and sends packets to same IP from another interface, that too is asymmetric routing.

Asymmetric Routing may lead to routing issues and should be avoided as far as possible. By default Red Hat Enterprise Linux 6 (and above) invalidate / discard packets when the route for outbound traffic differs from the route of incoming traffic i.e. asymmetrically routed packets are discarded.

But in certain cases you might require to enable it. The solution is to disable Strict Reverse Path Forwarding filtering (rp_filter)

rp_filter - INTEGER
0 - No source validation. (Enable Asymmetric Routing)
1 - Strict mode as defined in RFC3704 Strict Reverse Path
Each incoming packet is tested against the FIB and if the interface is not the best reverse path the packet check will fail.

By default failed packets are discarded.

Check present status:

# sysctl -a | grep '\.rp_filter'
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.bond0.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.eno1.rp_filter = 1
net.ipv4.conf.eno2.rp_filter = 1
net.ipv4.conf.eno3.rp_filter = 1
net.ipv4.conf.eno4.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 0

Value "1" means Strict Reverse Path Forwarding filtering is enabled So Assymmetric Routing is disabled

To make Temporary change and enable Assymmetric Routing:

# echo 0 > /proc/sys/net/ipv4/conf/default/rp_filter
# echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
# echo 0 > /proc/sys/net/ipv4/conf/bond0/rp_filter
# echo 0 > /proc/sys/net/ipv4/conf/eno1/rp_filter
# echo 0 > /proc/sys/net/ipv4/conf/eno2/rp_filter
# echo 0 > /proc/sys/net/ipv4/conf/eno3/rp_filter
# echo 0 > /proc/sys/net/ipv4/conf/eno4/rp_filter

Verify the changes:

# sysctl -a | grep '\.rp_filter'
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.bond0.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.eno1.rp_filter = 0
net.ipv4.conf.eno2.rp_filter = 0
net.ipv4.conf.eno3.rp_filter = 0
net.ipv4.conf.eno4.rp_filter = 0
net.ipv4.conf.lo.rp_filter = 0

To make Persistent change and enable Assymmetric Routing:

Add following lines in /etc/sysctl.conf

# sudo vi /etc/sysctl.conf
# Disable source validation by reversed path
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.bond0.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.eno1.rp_filter = 0
net.ipv4.conf.eno2.rp_filter = 0
net.ipv4.conf.eno3.rp_filter = 0
net.ipv4.conf.eno4.rp_filter = 0

Refresh with the new configuration
# sysctl -p

Reboot server
# sudo reboot

Now the Assymetric Routing is enabled.

CentOS7 it's done differently

Create a new conf file under the /etc/sysctl.d/ directory. File names take the format /etc/sysctl.d/[name].conf. Files in the /etc/sysctl.d/ directory are parsed in order so it is recommended to prepend the file name with a number signifying the order you would like the files to be parsed in. For example, /etc/sysctl.d/01-custom.conf

# sudo vi /etc/sysctl.d/01-enableAssymmetric.conf
# Disable source validation by reversed path
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.bond0.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.eno1.rp_filter = 0
net.ipv4.conf.eno2.rp_filter = 0
net.ipv4.conf.eno3.rp_filter = 0
net.ipv4.conf.eno4.rp_filter = 0

To have the system immediately apply the values in a new/updated /etc/sysctl.d file, run sysctl -p [filename]:

# sysctl -p /etc/sysctl.d/01-enableAssymmetric.conf