Using NFS Via an IPsec Tunnel

Peter McGarvey, <xaphod@freebsd-uk.eu.org>

Personally speaking, there are only two machines which I regard as mission-critical. The first is my home gateway, Green, which sits in my cellar. The second, Crisp, is my desktop at work. Both have stuff on them which I could need at any time, and SSH works fine for little things.

But it's the big things that cause a problem. I've got a considerable chunk of the O'Reilly zoo sitting on Green, whereas I usually need them when I'm working on Crisp. Then there is all the stuff which accumulates on Crisp which I'd really like to keep copies of on Green.

In the past I've replicated data. However, I've been known to reformat my desktop with the same relish (and for possibly the same reasons) as those who spend the whole weekend polishing the car. Besides, I hate the tediousness of squeezing stuff onto several CDs just to transport it . . . then ensuring both repositories stay in sync.

I have no problems with my laptop which lives on the private network behind Green. I just mount the relevant parts of Green via NFS and have a completely insecure web server. Whereas I could secure the web server, the only thing about NFS that anyone can agree on is that it's woefully insecure.

What I wanted was some way for Crisp to access my private network securely. So, one day I worked out just how to do it.

Here is what the network looks like:

       10.0.0.0/24 <-+-> green <--{ internet }--> crisp
                     |
    192.168.0.0/21 <-+

Green is an odd beastie, built out of bits of hardware I had lying around the house. It has 5 NICs because I had some dual port fxp's lying around and no hub/switch - hence all the 192 networks. Also, for some very good, logical, and totally forgotten reason, the web server and the NFS server listen on separate 10.0.0.x addresses which are bound as aliases to lo0. Anyway, that's all beside the point for this little mission.

The first thing I did was to create a tunnel between Green and Crisp. Forgetting all the weirdness, using fake addresses for Green and Crisp, this is what I wanted:

     green <-- 87.6.54.32 <--{ internet }--> 123.231.98.76 --> crisp
          ^                                                   ^
          |                                                   |
          +- 172.16.0.1 <--------------------> 172.16.0.2 -+

I'm using 172.16.0.0/30 for my endpoints for no other reason than that it's easy to spot this IP is different to my other internal IP addresses.

So, without further ado:

1. Create the tunnel.

       green> ifconfig gif0 create
       crisp> ifconfig gif0 create
       

2. Configure the physical source and destination addresses at each end.

       green> ifconfig gif0 tunnel 87.6.54.32 123.231.98.76
       crisp> ifconfig gif0 tunnel 123.231.98.76 87.6.54.32
       

3. Assign IP addresses to the gif devices on each machine. This is a point-to-point link, so the format is somewhat different from normal.

       green> ifconfig gif0 172.16.0.1 172.16.0.2 netmask 255.255.255.252
       crisp> ifconfig gif0 172.16.0.2 172.16.0.1 netmask 255.255.255.252
       

   Test it:

       green> ping 172.16.0.2
   
       PING 172.16.0.2 (172.16.0.2): 56 data bytes
       64 bytes from 172.16.0.2: icmp_seq=0 ttl=64 time=40.727 ms
       64 bytes from 172.16.0.2: icmp_seq=1 ttl=64 time=29.629 ms
   
       crisp> ping 172.16.0.1
       64 bytes from 172.16.0.1: icmp_seq=0 ttl=64 time=28.884 ms
       64 bytes from 172.16.0.1: icmp_seq=1 ttl=64 time=41.726 ms
       

   A bit of tcpdumpage on the public interface:

       12:42:25.172376 green > crisp: 172.16.0.1 > 172.16.0.2: icmp: echo request (ipip-proto-4)
       12:42:25.172576 crisp > green: 172.16.0.2 > 172.16.0.1: icmp: echo reply (ipip-proto-4)
       12:42:26.182762 green > crisp: 172.16.0.1 > 172.16.0.2: icmp: echo request (ipip-proto-4)
       12:42:26.182952 crisp > green: 172.16.0.2 > 172.16.0.1: icmp: echo reply (ipip-proto-4)
       12:42:27.196408 green > crisp: 172.16.0.1 > 172.16.0.2: icmp: echo request (ipip-proto-4)
       

4. Set up routing.

       crisp> route add 10/24 -interface gif0
       crisp> route add 192.168.0.0/21 -interface gif0
       

   Test it:

       crisp> ping 192.168.3.100
   
       PING 192.168.3.100 (192.168.3.100): 56 data bytes
       64 bytes from 192.168.3.100: icmp_seq=0 ttl=63 time=33.278 ms
       64 bytes from 192.168.3.100: icmp_seq=1 ttl=63 time=32.698 ms
       

5. Save the configuration in /etc/rc.conf.

   Adding this on Green:

       gif_interfaces="gif0"
   
       gifconfig_gif0="87.6.54.32 123.231.98.76"
       ifconfig_gif0="gif0 172.16.0.1 172.16.0.2 netmask 255.255.255.252"
       

   And this on Crisp:

       gif_interfaces="gif0"
   
       gifconfig_gif0="123.231.98.76 87.6.54.32"
       ifconfig_gif0="gif0 172.16.0.2 172.16.0.1 netmask 255.255.255.252"
   
       static_routes="gif0_0 gif0_1"
       route_gif0_0="10/24 -interface gif0"
       route_gif0_1="192.168.0.0/21 -interface gif0"
       

The next thing I needed to do was make this tunnel secure. I personally find IPsec a bit confusing. I suppose if I actually read the man pages properly I could clear up this confusion. A quick skim and a bit of Googling, however, is much more fun. Looking at the rc scripts in /etc also proved useful. So, eventually, I figured out what was required.

The first requirement was to recompile my kernel after adding these two lines to the configuration:

    options         IPSEC
    options         IPSEC_ESP
    

I could have included this too:

    options         IPSEC_DEBUG
    

I have little idea though as to what additional functionality this gives me, so I didn't.

Next, I needed something to manage the key exchange. The package most examples seemed to use was racoon. Installing and configuring it proved remarkably easy:

    green> cd /usr/ports/security/racoon
    green> make install clean
    green> cd /usr/local/etc/racoon
    green> echo "172.16.0.2 my_shared_secret" > psk.txt
    green> chmod 0600 psk.txt
    green> cp racoon.conf.dist racoon.conf

    crisp> cd /usr/ports/security/racoon
    crisp> make install clean
    crisp> cd /usr/local/etc/racoon
    crisp> echo "172.16.0.1 my_shared_secret" > psk.txt
    crisp> chmod 0600 psk.txt
    crisp> cp racoon.conf.dist racoon.conf
    

The sample configuration is alright to use as is; it just needs one small change. Add "isakmp 172.16.0.x [500];" to the "listen" block, where x->1 on green, and x->2 on crisp.

So on crisp it looks like this:

    listen
    {
    isakmp 172.16.0.2 [500];
    }
    

It's only a moment's work to set syslog to log racoon messages to /var/log/racoon.log (not forgetting newsyslog, of course). Firing racoon up is simple:

    crisp> /usr/local/etc/rc.d/racoon.sh start

    green> /usr/local/etc/rc.d/racoon.sh start
    

With no warnings, I think it's safe to assume this means it's all fine.

Finally, I need to get the kernel to encrypt my packets. This really can be confusing. I also discovered a mild gotcha.

The command to use is setkey; the configuration file is /etc/ipsec.conf. Add this to /etc/rc.conf:

    ipsec_enable="YES"
    

And you can clear the entries and apply manually with:

    setkey -F -FP
    setkey -f /etc/ipsec.conf
    

However, getting /etc/ipsec.conf right really was a case of trial and error. Thanks to some long-forgotten experimenting, Green has some weird port redirection going on which broke with my first attempt. So eventually I ended up with this:

On crisp:

    flush;
    spdflush;
    spdadd 172.16.0.2/32 172.16.0.1/32 any -P out ipsec esp/tunnel/172.16.0.2-172.16.0.1/require;
    spdadd 172.16.0.1/32 172.16.0.2/32 any -P in ipsec esp/tunnel/172.16.0.1-172.16.0.2/require;

    spdadd 172.16.0.2/32 10.0.0.0/24 any -P out ipsec esp/tunnel/172.16.0.2-172.16.0.1/require;
    spdadd 10.0.0.0/24 172.16.0.2/32 any -P in ipsec esp/tunnel/172.16.0.1-172.16.0.2/require;

    spdadd 172.16.0.2/32 192.168.0.0/21 any -P out ipsec esp/tunnel/172.16.0.2-172.16.0.1/require;
    spdadd 192.168.0.0/21 172.16.0.2/32 any -P in ipsec esp/tunnel/172.16.0.1-172.16.0.2/require;
    

On green:

    flush;
    spdflush;

    spdadd 172.16.0.1/32 172.16.0.2/32 any -P out ipsec esp/tunnel/172.16.0.1-172.16.0.2/require;
    spdadd 172.16.0.2/32 172.16.0.1/32 any -P in ipsec esp/tunnel/172.16.0.2-172.16.0.1/require;

    spdadd 10.0.0.0/24 172.16.0.2/32 any -P out ipsec esp/tunnel/172.16.0.1-172.16.0.2/require;
    spdadd 172.16.0.2/32 10.0.0.0/24 any -P in ipsec esp/tunnel/172.16.0.2-172.16.0.1/require;

    spdadd 192.168.0.0/21 172.16.0.2/32 any -P out ipsec esp/tunnel/172.16.0.1-172.16.0.2/require;
    spdadd 172.16.0.2/32 192.168.0.0/21 any -P in ipsec esp/tunnel/172.16.0.2-172.16.0.1/require;
    

These "spadd" entries pair into three groups: (1) traffic between end-points, (2) between crisp and my virtual network on lo0, (3) between crisp and all green's fxp's. The setkey(8) man page does provide enlightenment, but not so much as I can explain it. On the other hand, a quick tcpdump of the public interface on crisp when I'm pinging from green proves it's working:

    21:55:38.389071 crisp > green: 172.16.0.2 > 172.16.0.1: ESP(spi=0x006d5c08,seq=0xf) (ipip-proto-4)
    21:55:38.389938 green > crisp: 172.16.0.1 > 172.16.0.2: ESP(spi=0x088348ef,seq=0xe) (ipip-proto-4)
    21:55:39.399602 crisp > green: 172.16.0.2 > 172.16.0.1: ESP(spi=0x006d5c08,seq=0x10) (ipip-proto-4)
    21:55:39.400492 green > crisp: 172.16.0.1 > 172.16.0.2: ESP(spi=0x088348ef,seq=0xf) (ipip-proto-4)
    

Now I can mount bits of Green on Crisp without any worries. However, I've just read that a new version of NFS is just around the corner which is secure. At least I don't have to secure my web server either.

References

http://www.freebsddiary.org/

As ever, the FreeBSD Diary is always worth a look. There are a couple of relevant entries.

http://rr.sans.org/firewall/IPSec_VPN.php

http://asherah.dyndns.org/~josh/ipsec-howto.txt

Both of these were useful, but there was a little too much background detail which I just skimmed.