Multiple default routes in FreeBSD without BGP or similar

From Wiki
Jump to: navigation, search

Normally, something like BGP would be used to fail over between two WAN links. This has many advantages, including the ability to retain the same IP addresses used on each link, even across diverse providers. Of course, as a consumer, things like BGP are not readily available from your ISP. Thankfully, FreeBSD has the capability to provide poor man's failover using multiple routing tables.

My setup looks a bit like this:

        ADSL
         |
+------------------+
|  FreeBSD router  | --- 3G
+------------------+
         ||
  +-+-+--++--+-+-+
  | | |      | | |
   Various  VLANs

The default route is via the ADSL connection for the default routing table, FIB0. FIB1, the second routing table has a default route via the 3G connection. When the ADSL connection is down, pf is reconfigured to send essential traffic through the 3G interface until ADSL service is restored.

How?

The kernel needs to be rebuilt with support for multiple routing tables. Even if you have not done this before, it is very simple, see: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-building.html

I appended this to my kernel configuration:

# Multiple routing tables
options         ROUTETABLES=2

Clearly, you can configure as many routing tables as you like, but two is enough for me. Then wait ages for the kernel to rebuild and reboot to load the new kernel:

# cd /usr/src
# make buildkernel KERNCONF=MYKERNEL && make installkernel KERNCONF=MYKERNEL
# shutdown -r now

With the new kernel running, you'll have the ability to choose the routing table you want to operate on by prepending setfib 0 or setfib 1 to your command:

# setfib 0 netstat -rn | grep default
default            192.0.2.1        UGS         0    92355    vr0

# setfib 1 netstat -rn | grep default
default            192.0.2.1        UGS         0    92355    vr0

So, we can alter the second routing table to have a different default gateway:

# setfib 1 route delete default
# setfib 1 route add default 192.0.3.1

Observe that the default route has changed for FIB 1, traffic will be sent to a different gateway on interface vr3:

# setfib 0 netstat -rn | grep default
default            192.0.2.1        UGS         0    92355    vr0

# setfib 1 netstat -rn | grep default
default            192.0.3.1        UGS         0    92355    vr3

Test this:

# setfib 0 ping -c4 google.com
PING google.com (173.194.34.161): 56 data bytes
64 bytes from 173.194.34.161: icmp_seq=0 ttl=57 time=28.486 ms
64 bytes from 173.194.34.161: icmp_seq=1 ttl=57 time=28.379 ms
64 bytes from 173.194.34.161: icmp_seq=2 ttl=57 time=28.442 ms
64 bytes from 173.194.34.161: icmp_seq=3 ttl=57 time=28.237 ms


--- google.com ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 28.237/28.386/28.486/0.094 ms
# setfib 1 ping -c4 google.com
PING google.com (173.194.34.174): 56 data bytes
64 bytes from 173.194.34.174: icmp_seq=0 ttl=53 time=84.300 ms
64 bytes from 173.194.34.174: icmp_seq=1 ttl=53 time=111.985 ms
64 bytes from 173.194.34.174: icmp_seq=2 ttl=53 time=89.524 ms
64 bytes from 173.194.34.174: icmp_seq=3 ttl=53 time=66.979 ms

--- google.com ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 66.979/88.197/111.985/16.070 ms

So, we have at least ICMP connectivity through each interface. If this is working to your satisfaction, use rc.local to apply this configuration on each boot:

# vi /etc/rc.local

# Set default route for fib 1 (3G interface)
setfib 1 route delete default
setfib 1 route add default 192.0.3.1

Now, pf can be told to use either FIB 0 or FIB 1 for any given pass in rule. A simplified configuration might be:

int_if = "vr1"
ext_if = "vr0"
3g_if = "vr3"

table <adsl> persist { 192.168.0.0/24, 192.168.1.0/24 }
table <3g> persist

# NAT
nat on $ext_if from { 192.168.0.0/24, 192.168.1.0/24 } to any -> ($ext_if)
nat on $3g_if from { 192.168.0.0/24, 192.168.1.0/24 } to any -> ($3g_if)

# Filter rules
pass all
pass in from <adsl> to any rtable 0
pass in from <3g> to any rtable 1

So, by default, all traffic from the 192.168.0/24 and 192.168.1/24 networks are dealt with by FIB 0, as specified by the rtable 0 option. Using pfctl to manipulate the <3g> and <adsl> tables will cause traffic from different hosts to be routed through the desired gateway:

# pfctl -t adsl -T delete 192.168.0.0/24
# pfctl -t 3g -T add 192.168.0.0/24

Because this can be done without the need to edit and reload pf.conf, this makes it easy to alter the routing automatically from scripts, for example in response to an interface going down or certain internet hosts becoming unreachable.