IPv6+Xen on a Hetzner server with routing to dummy0 and proxy ndp

From Wiki
Jump to: navigation, search

Overview

My setup for Xen domU networks is like so:

                                    dummy0 
                                192.0.2.0/24 
                                    ^  |
                                    |  |
                                    |  v
                         bridge --> dom0 --> eth0 --> the network
                           ^
                           |
domU 1:    eth0 -> vif1.0 -+
                           |
domU 2:    eth0 -> vif2.0 -+
                           |
domU 3:    eth0 -> vif3.0 -+
                           |
domU 4:    eth0 -> vif4.0 -+

The fictitious 192.0.2.0/24 network is routed by Hetzner to my dom0's primary eth0 IP address. I assign 192.0.2.1 to the interface dummy0 and bridge this to the Xen domUs. This gives me greater control over what the domUs can do, as some of these virtual machines are not administered by me. See this page about networking for untrusted domUs for full details.

This approach does not work for IPv6 addresses. Hetzner assign me a /64, but there is no /126 interceding network and a /64 must not be further subnetted. It is expected that the domUs will be in the same layer 2 segment as Hetzner's IPv6 router, which clearly isn't the case for me. The solution is to use the IPv6 equvalent of proxy arp, which is called proxy ndp.

                                    dummy0 
                             2001:DB8:3:2:1::1/64 
                                    ^  |
                                    |  |
                                    |  v
                         bridge --> dom0 ----------> eth0 ------------> the network
                           ^                 2001:DB8:3:2:1::1/64
                           |
                           |
domU 1:    eth0 -> vif1.0 -+
    2001:DB8:3:2:1::10/64  |
                           |
domU 2:    eth0 -> vif2.0 -+
    2001:DB8:3:2:1::11/64  |
                           |
domU 3:    eth0 -> vif3.0 -+
    2001:DB8:3:2:1::12/64  |
                           |
domU 4:    eth0 -> vif4.0 -+
    2001:DB8:3:2:1::13/64

So, the dom0 has the (fictitious) IPv6 address 2001:DB8:3:2:1::1/64 assigned to both eth0 and dummy0. proxy_ndp will handle ndp annoucements on behalf of the domUs, so it will not matter to the upstream IPv6 router that the domUs are not directly connected.

Assigning IPs and Setting up proxy_ndp

# ip -6 addr add 2001:DB8:3:2:1::1/64 dev eth0
# ip -6 addr add 2001:DB8:3:2:1::1/64 dev dummy0

Hetzner has an oddity where the default route is outside the allocated subnet. In my made up example, our allocation 2001:DB8:3:2:1::/64 is routed by 2001:DB8:3:2::1 (note carefully and understand the difference). You'll need to check the specifics for your own allocation, but it will be like so:

# ip -6 route add 2001:DB8:3:2::1 dev eth0
# ip -6 route add default via 2001:DB8:3:2::1 dev eth0


Now, you should already be able to ping6 something from the dom0

# ping6 -n -c4 www.google.co.uk 
PING www.google.co.uk(2a00:1450:4016:800::1017) 56 data bytes
64 bytes from 2a00:1450:4016:800::1017: icmp_seq=1 ttl=55 time=15.8 ms
64 bytes from 2a00:1450:4016:800::1017: icmp_seq=2 ttl=55 time=15.4 ms
64 bytes from 2a00:1450:4016:800::1017: icmp_seq=3 ttl=55 time=16.4 ms
64 bytes from 2a00:1450:4016:800::1017: icmp_seq=4 ttl=55 time=15.3 ms
4 packets transmitted, 4 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 15.318/32.972/101.729/34.380 ms

Next, alter the routing table so that the dom0 realises that, with the exception of its own address, all addresses in our allocated range 2001:DB8:3:2:1::/64 are to be found on dummy0 rather than eth0:

# ip -6 route del 2001:DB8:3:2:1::/64 dev eth0

So, this is what we want to see:

# ip -6 route show | grep 2001:DB8
2001:DB8:3:2::1 dev eth0  metric 1024  mtu 1500 advmss 1440 hoplimit 4294967295
2001:DB8:3:2:1::/64 dev dummy0  metric 1024  mtu 1500 advmss 1440 hoplimit 4294967295
default via 2001:DB8:3:2::1 dev eth0  metric 1024  mtu 1500 advmss 1440 hoplimit 4294967295

Again, note the distinction between 2001:DB8:3:2:1::/64 and 2001:DB8:3:2::1. Yes I know this is a horrible example, but sometimes it isn't immediately obvious that your gateway is in another /64 within the same /48, so you must look carefully.

Now, to actually enable proxy_ndp. Insert this at the end of /etc/sysctl.conf:

## Settings necessary for IPv6 proxy ndp (like proxy arp)
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.proxy_ndp = 1
net.ipv6.conf.all.proxy_ndp = 1

And read in the changes:

# sysctl -p

Unlike proxy_arp, this still doesn't just work; it's necessary to manually specify the IPv6 address that you wish to proxy. This isn't difficult:

# ip -6 neigh add proxy 2001:DB8:3:2:1::10 dev eth0
# ip -6 neigh add proxy 2001:DB8:3:2:1::10 dev dummy0

Finally, add an address and a default route to the domU:

# ip -6 addr add 2001:DB8:3:2:1::10 dev eth0
# ip -6 route add default via 2001:DB8:3:2:1::1

proxy_ndp for an Entire /64 Subnet

I don't want to be forced to add individual NDP proxy entries for every domU host, so I set up ndppd to handle the whole /64.

# wget http://www.priv.nu/projects/ndppd/files/ndppd_0.2.2-1_amd64.deb
# dpkg -i ndppd_0.2.2-1_amd64.deb

The config file is very well commented, so let's start with that:

# cp /usr/share/doc/ndppd.conf-dist /etc/ndppd.conf

You'll need at least:

# vi /etc/ndppd.conf

route-ttl 30000
proxy eth0 {
   router yes
   timeout 500   
   ttl 30000
   rule 2001:DB8:3:2:1::/64 {
      auto
   }
}

Start ndppd and you're away:

# /etc/init.d/ndppd start

Security

Although IPv6 connectivity is far from universal, we still need to be careful not to offer any unnecessary services.

To protect the dom0:

# vi /etc/ip6tables.rules

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i lo -j ACCEPT 
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Allow SSH connections for management
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
# ICMP is nice sometimes
-A INPUT -p ipv6-icmp -j ACCEPT 
-A INPUT -j DROP 
COMMIT

Apply the configuration:

# ip6tables-restore < /etc/ip6tables.rules

Update the network settings so that it is applied at boot time:

# vi /etc/network/interfaces

iface eth0 inet6 static
 address 2001:DB8:3:2:1::1
 netmask 64
 # Activate the ip6tables rules at boot time
 pre-up ip6tables-restore < /etc/ip6tables.rules

Because proxy_ndp takes place before the above ip6tables rules get involved, this won't interfere with the traffic heading to and from the domUs, but it won't protect them either. Make sure you use a similar ip6tables setup on those, too.

Tidying up

We need to make sure that the necessary routing table and address assignment happens when the dom0 reboots. This is all accomplished from /etc/network/interfaces:

# vi /etc/network/interfaces

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
 address 192.0.1.3
 netmask 255.255.255.192
 gateway 192.0.1.1
 pre-up iptables-restore < /etc/iptables.rules

iface eth0 inet6 static
 address 2001:db8:3:2:1::1
 netmask 64
 # Add our default gateway
 up ip -6 route add 2001:db8:3:2::1 dev eth0
 down ip -6 route del 2001:db8:3:2::1 dev eth0
 up ip -6 route add default via 2001:db8:3:2::1 dev eth0
 down ip -6 route del default via 2001:db8:3:2::1 dev eth0
 # Remove the route for our /64 so it is only reachable on dummy0
 up ip -6 route del 2001:DB8:3:2:1::/64 dev eth0
 # Activate the ip6tables rules at boot time
 pre-up ip6tables-restore < /etc/ip6tables.rules
 # Listen for packets destined for other ARP/NDP addresses
 up ifconfig eth0 promisc

auto dummy0
iface dummy0 inet static
 address 192.0.2.1
 netmask 255.255.255.0

iface dummy0 inet6 static
 address 2001:db8:3:2:1::1
 netmask 64
 # Listen for packets destined for other ARP/NDP addresses
 up ifconfig dummy0 promisc