Building an ATM Firewall with BSD

By Martin Wellard veg@squirrel.co.uk

Introduction

As a system administrator in a large academic institution, it is obvious to me that network security should be a high priority when planning a network strategy. However, it's not the system administrators that make the decisions and while most managers are willing to pay lip service to the need for security, they are far less willing to actually start writing cheques.

After a couple of unsuccessful crack attempts (and a couple of partially successful ones) we were able to justify a firewall. This in itself was an achievement. However, even at the best of times, the world of UK academia moves at a snail's pace and I for one wanted something in place very quickly.
 

Starting out

Our network consists of an extremely expensive ATM backbone running Lan-Emulation (LANE) out to switched 10M ports. The connection to the Internet consists of a single CLIP (Classical IP) PVC (Permanent Virtual Circuit) over a 34M ATM link from a metropolitan area network. Once inside our network, the PVC enters our central ATM switch, where it is converted to 155M LANE.

Our ATM vendors suggested that the best solution for us would be to use Firewall-1 on NT with two ATM adapters.

Personally, I dislike ATM as a way of carrying IP, and dislike LANE even more. There are plenty of reasons for this that run beyond the scope of this article but I was keen to avoid its use in a firewall and the thought of a firewall on NT turns the blood cold. So it was time to take action. We discussed various possible methods of creating an ethernet level "gap" between our external connection and the main part of the network, which would allow us to implement a nice bridging firewall with OpenBSD. However, after many discussions it became apparent that our network equipment was incapable of letting us do this, and ATM was the only way around it.

Anxious to avoid becoming ensconced in a proprietary quagmire, I suggested buying an ATM adapter and testing OpenBSD's built-in, open, firewalling capabilities. Sadly the ATM support in OpenBSD was only able to work with two families of ATM adapter, both of which were difficult to procure. The only cards we could easily obtain were the ones recommended by our vendors, the FORE 200e, which only provided drivers for SPARC Solaris and NT. Whilst I was aware that Linux also supported these cards, the idea of having core equipment running pre-alpha software did not inspire confidence.

It was looking like all was lost and we would yet again have to capitulate to the proprietary dictators when by accident I noticed a mention of the FORE ATM adapters on the FreeBSD site under a different ATM subsystem known as HARP. After some rapid research it became apparent that FreeBSD would support it perfectly well.

So, we ordered two adapters (one as spare) and I set about installing FreeBSD on a Pentium II 433 we had put aside for the purpose. Half an hour later, the box was up and running with the ATM adapter and a 3Com ethernet adapter installed. Ideally I would have preferred to use an Intel EtherExpress Pro as past experience has made me lose confidence in 3Com adapters under high load.

Kernel Configuration

The next step was to build a kernel that recognised them. Luckily, being BSD, there was plenty of useful documentation on the ATM subsystem contained in man pages and in /usr/share/examples/atm.

Using a copy of the GENERIC kernel config , I commented out pretty much everything except for essentials and added the following from LINT:

options ATM_CORE                #core ATM protocol family
options ATM_IP                  #IP over ATM support
options ATM_SIGPVC              #SIGPVC signalling manager
options ATM_SPANS               #SPANS signalling manager
options ATM_UNI                 #UNI signalling manager
device  hfa0                    #FORE PCA-200E ATM PCI

Also, as this machine was eventually going to be firewalling I added support for ipfilter:
 

options IPFILTER

options IPFILTER_LOG

The choice of ipfilter over ipfirewall was purely personal. I'd used ipfilter before on OpenBSD and liked it.

Additionally, I upped MAXUSERS and the number of mbuf clusters so that high levels of traffic wouldn't be a problem:

maxusers 128
options NMBCLUSTERS 4096

The kernel was compiled, installed and booted. It came up cleanly, and obediently recognised the FORE adapter.

ATM configuration

The FORE 200e adapters require microcode to be uploaded every time that it is powered up. The documentation in /usr/share/examples/atm provided detailed instructions on obtaining the binary code from the FORE web-site and installing it. The binary itself consists of a small file called pca200e.bin which is uploaded to the card using the command fore_dnld, a standard part of the FreeBSD distribution. In true, friendly, BSD-style the rc scripts were already ATM aware and very little needed to be altered to configure the subsystem including uploading the microcode.

In order to describe the workings of the HARP ATM subsystem, here are the options in /etc/defaults/rc.conf for our setup. After each option is a description of what it does

atm_enable="YES"

Causes the rc.atm to be executed with the ATM settings contained in rc.conf. The pca200e.bin file, if present in /etc will be uploaded to all installed adapters automatically at boot if this is set to YES.

atm_netif_hfa0="fa 1"

Each Fore 200e has a "physical" interface name consisting of "hfa" followed by a number. The first interface is hfa0, the second hfa1 and so on. These interfaces are unusual as they don't correspond to the normal network interfaces such as xl0 or ppp0. Instead, each adapter must have one or more "logical" interfaces (confusingly referred to in the atm documentation as a "netif") which can be given their own IP addresses and be manipulated as normal interfaces with ifconfig and route etc.

The user decides the names given to these virtual interfaces. In the configuration line above, I have stated that I wish the interfaces to be called "fa", and only one to be initialised. This interface will therefore be called "fa0".

If we had wanted multiple logical interfaces for the ATM adapter, we could have done something like: atm_netif_hfa0="fa 5" which would have created 5 "netif"s called fa0, fa1,fa2,fa3 and fa4. Each of these could have then been given a different IP address with ifconfig.

atm_sigmgr_hfa0="sigpvc" # Signalling manager for physical interface.

This line indicates that we will only be using PVCs with this interface. ATM has many modes of operation and each interface must be bound to a "signalling manager" that controls how it will be used.

The HARP system supports four different signal managers, all of which support PVCs.
 

atm_macaddr_hfa0="NO" # Override physical MAC address (or NO).

Each 200e comes with its own MAC address, however this line allows you to override this with an alternative. We'll stick to the factory preset.

#atm_prefix_hfa0="ILMI" # NSAP prefix (UNI interfaces only) (or ILMI).

#atm_arpserver_atm0="0x47.0005.80.999999.9999.9999.9999.999999999999.00" # ATMARP server address (or local).

#atm_scsparp_atm0="NO" # Run SCSP/ATMARP on network interface (or NO).

These are commented out as we are not using UNI and thus don't need an ATM address. The scope of this article doesn't allow a full description of these.

atm_pvcs="lman"

Here we have created a whitespace separated list of PVCs that we define further down. In our case we only have one.

atm_arps=""

Again, we don't need to use ATMARP as we are using PVCs.

atm_pvc_lman="hfa0 0 126 aal5 snap ip fa0 AAA.BBB.CCC.DDD"

Here is where we define our pvc which is arbitrarily called 'lman'. The content consists of arguments to the command atm pvc. In a nutshell, this line may be read as:
"Create a PVC on hfa0 with VPI 0, VCI 26 using AAL5 with SNAP headers. It will carry IP via interface fa0, and the address at the far end of the PVC is AAA.BBB.CCC.DDD".
The values of the VPI and VCI were provided by the administrators of the metropolitan area network.
See atm(8) for a full description of the arguments used with the atm command.

Testing

With this being such a core piece of equipment, we wanted to satisfy ourselves that it could stand up to the job. We also wanted to ensure that the ATM side of things was going to work nicely with our network and with ipfilter. So as an initial test, we constructed an identical FreeBSD box with the spare ATM adapter. Both boxes were then connected to each other via the ATM switch, and a PVC between them established.

It wasn't so much that we didn't expect it work, more that we didn't expect it to work the first-time. But it did. Perfectly. Several nasty traffic generation scripts were then constructed and executed on both machines to really hammer the link, and everything continued to work perfectly. So we then started to play around with ipfilter to ensure it co-operated with the ATM system properly. Again, it worked perfectly no matter how harshly we treated it.
 

Implementation

Now that we had satisfied ourselves that it was all going to work properly the next stage was to decide on a method of implementation. It was decided that we would initially run it in place with no firewalling for a couple of days, to ensure that it was happy with the bandwidth. After this we could slowly introduce rules.

Due to our network configuration, the firewall would effectively become the border router, routing between the CLIP PVC and our LANE backbone. Beforehand, routing was achieved by one of the FORE Powerhubs which formed part of the LANE network.
The ethernet adapter plugged directly into one of the switched LANE ports, and the ATM adapter connected to the incoming PVC via the main ATM switch (this was necessary as the incoming connection was 34Mb and the switch allowed us to convert it to standard 155Mb).

We closely monitored all traffic for a couple of days until we were sure there was no detrimental effect on bandwidth after the implementation of the firewall. It could have been my imagination, but it actually seemed to improve performance. Regardless, there was certainly no noticeable reduction in throughput.

The next stage was to implement a set of light rules that would block some really obvious holes such as IP spoofing and smurfing. We based our initial ruleset around the example sets provided as part of the OpenBSD distribution in /usr/share/ipf (available from the OpenBSD web site). Once we were happy with our rules, we introduced them and monitored the filter with ipmon(8). In fact we did notice a few legitimate packets being blocked and traced the problem to a small typo in the rules file. It was a simple matter to fix and of course no interruption in service was required to do so.

Conclusion

We have been running the firewall in place for several months now (without a reboot - it's not NT), with the "light" firewalling rules in place and have had no problems. The throughput is equally as good as before and we frequently have periods of extremely high load which FreeBSD seems to cope with admirably. We have also observered (with ipmon) many legitimate blocks of scans and unauthorised NFS mount attempts.

We have delayed instituting firmer rules until we are sure we have accounted for all internal machines that require direct access to the outside world. Indeed, the political aspects of this project proved more of an obstacle than either technical or financial constraints. The process of auditing all machines in the institution has highlighted just how many ad-hoc servers have been established, independently of the computer services department, on which people have begun to rely. The firewall is also providing us with a perfect opportunity to audit and rationalise them.

Overall FreeBSD has provided us with a quick, powerful, and secure solution. Past experiences with other commercial firewalls has shown me that security is certainly not proportional to price. Knowing exactly what's going on in the FreeBSD box gives me far greater confidence in the security of the firewall than I've had with any other proprietary product.