Free Redundant Layer 3 Routing with VRRP 802.1Q VLANs using Debian Linux

How to use Linux to get redundant routing configured, with enterprise grade features.

Distro: Debian 6 Server

Install a base Debian 6 server, specific to your environment. The first NIC should be configured with your preferred default route, eg. firewall. We select the default packages "Standard" and "SSH".

If using VMware, create a new vSwitch in VLAN "All" (802.1Q Tag 4095)

Add a second network adapter to the Virtual Machine, connected to the new vSwitch.

apt-get install vlan

Add the VLANs as per the diagram above.

vconfig add eth1 70

vconfig add eth1 80

vconfig add eth1 90

Configure the network interfaces file /etc/network/interfaces using your favourite text editor (mine's nano) with these additions:

auto eth0
auto eth1.70
iface eth1.70 inet static
        address 192.168.70.253
        netmask 255.255.255.0
        network 192.168.70.0
        broadcast 192.168.70.255
        vlan_raw_device eth1
auto eth1.80
iface eth1.80 inet static
        address 192.168.80.253
        netmask 255.255.255.0
        network 192.168.80.0
        broadcast 192.168.80.255
        vlan_raw_device eth1
auto eth1.90
iface eth1.90 inet static
        address 192.168.90.253
        netmask 255.255.255.0
        network 192.168.90.0
        broadcast 192.168.90.255
        vlan_raw_device eth1

Restart networking: /etc/init.d/networking restart.

Enable IP forwarding (routing):

echo 1 > /proc/sys/net/ipv4/ip_forward

Make the change permanent, uncomment this line in /etc/sysctl.conf

#net.ipv4.ip_forward = 1

Install a DHCP server

apt-get install isc-dhcp-server

Remove the default file /etc/dhcp/dhcpd.conf and replace with:

log-facility local7;
subnet 192.168.70.0 netmask 255.255.255.0 {
  range 192.168.70.20 192.168.70.25;
  option domain-name-servers 192.168.70.253,192.168.70.252;
  option domain-name "example.int";
  option routers 192.168.70.254;
  option broadcast-address 192.168.70.255;
  default-lease-time 2400;
  max-lease-time 7200;
}
subnet 192.168.80.0 netmask 255.255.255.0 {
  range 192.168.80.20 192.168.80.25;
  option domain-name-servers 192.168.80.253,192.168.80.252;
  option domain-name "example.int";
  option routers 192.168.80.254;
  option broadcast-address 192.168.80.255;
  default-lease-time 2400;
  max-lease-time 7200;
}
subnet 192.168.90.0 netmask 255.255.255.0 {
  range 192.168.90.50 192.168.90.150;
  option domain-name-servers 192.168.90.253,192.168.90.252;
  option domain-name "example.int";
  option routers 192.168.90.254;
  option broadcast-address 192.168.90.255;
  default-lease-time 2400;
  max-lease-time 7200;
}

Start the DHCP server:

/etc/init.d/isc-dhcp-server start

Install a DNS Caching Only Server

 apt-get install bind9

Edit /etc/bind/named.conf.options and add these lines after the comments //
        listen-on { any; };

        forwarders {<your DNS forwarder1>;<your DNS forwarder2>;};

Start BIND:

/etc/init.d/bind9 start

Now shutdown your VM (or Physical) and clone it to another physical server.

Boot the 'clone, and change the hostname/IP in the following files:

/etc/hostname [change from RouterPri to RouterBak]

/etc/hosts [change from RouterPri to RouterBak]

/etc/network/interfaces [Change IPs from .253 to .252]

/etc/dhcp/dhcpd.conf [Change the IP pools to be different to the primary]

Reboot, Boot the 'original'

Install 'keepalived' and 'vrrpd'

apt-get install keepalived vrrpd -y

Configure on RouterPri file /etc/keepalived/keepadlived.conf

vrrp_instance VR1 {
        state MASTER
        interface eth1
        virtual_router_id 1
        priority 100
        authentication {
        auth_type PASS
        auth_pass password
        }
        virtual_ipaddress {
        192.168.70.254/24 brd 192.168.70.255 dev eth1.70
        192.168.80.254/24 brd 192.168.80.255 dev eth1.80
        192.168.90.254/24 brd 192.168.90.255 dev eth1.90
        }

Configure the same file on RouterBak

vrrp_instance VR1 {
        state SLAVE
        interface eth1
        virtual_router_id 1
        priority 50
        authentication {
        auth_type PASS
        auth_pass password
        }
        virtual_ipaddress {
        192.168.70.254/24 brd 192.168.70.255 dev eth1.70
        192.168.80.254/24 brd 192.168.80.255 dev eth1.80
        192.168.90.254/24 brd 192.168.90.255 dev eth1.90
        } 

At this point, you can assign a NIC to one of your other VMs (using VMXNET3)

and use VLAN tags to test DHCP, DNS and Primary to Backup L3 failover. 

A dynamic routing protocol is required to notify other Layer 3 devices on the

network of the route change, if failover is to occur. For this, we will use the 'quagga'

daemon and configure it with the RIPv2 protocol.

apt-get install quagga

Edit /etc/quagga/daemons

zebra=yes

ripd=yes

Edit /etc/quagga/zebra.conf

hostname Router
password zebra
enable password zebra
!
interface eth0
 ipv6 nd suppress-ra
!
interface eth1
 ipv6 nd suppress-ra
!
interface eth1.70
 ipv6 nd suppress-ra
!
interface eth1.80
 ipv6 nd suppress-ra
!
interface eth1.90
 ipv6 nd suppress-ra
!
interface lo
!
ip forwarding
!
!
line vty
! 

Edit /etc/quagga/ripd.conf on both routers.

hostname Router*.example.int
password zebra
log file /var/log/quagga/ripd.log
log stdout
!
router rip
 version 2
 timers basic 30 120 120
 redistribute kernel
 redistribute connected
 redistribute static
 network 192.168.70.0/24
 network 192.168.80.0/24
 network 192.168.90.0/24
 network eth0
 network eth1
 network eth1.70
 network eth1.80
 network eth1.90
 neighbor <Your L3 Device 1>
 neighbor <Your L3 Device 2>
!
line vty
!
Start the quagga daemon: /etc/init.d/quagga start

You can now test the failover/failback of your HA solution by disconnecting and 

reconnecting NICs on the Primary router, and monitoring /var/log/syslog

You should see only a few packets dropped (depending on the convergence of

other L3 devices) during the failover and failback between the routers.

Your syslog during failover/failback should look something like this:

Thoughts on running this in production...

• Consider installing each router on the local disks of your VMware hosts

  • Set them to auto-boot with the hosts 

    Ensure you can communicate on the same subnet/VLAN in case of problems 

Please let me know your thoughts and feedback if you've found this useful!