The New Link-Level Independent ARP Subsystem of NetBSD

Ignatios Souvatzis, The NetBSD Project

Abstract

However, when dealing with other types of addresses like ARCnet, AX25 packet radio, etc. with a different length, more general aspects of the ARP mapping have to be implemented. This paper reports on the one-to-N mapping developed for NetBSD.

1.  The Problem

When transmitting network packets over a non-point-to-point link, the network address must be somehow translated to the link level address. For IP version 4, this is normally done using the Address Resolution Protocol [1]. Contrary to its name, it is designed to translate any network protocol address type to any link layer address type (although it only has been in use for IP version 4 and for Ethertalk, both over Ethernet.

Old BSD networking up to and including 4.4BSD-lite has only supported IP version 4 over Ethernet. All modern operating systems based on 4.4BSD-lite have inherited this limitation. The exception are link types that use the same 6-byte addressing as Ethernet (e.g., FDDI).

If you lie to the above layer about the address length, it should even be possible to support link types with an address length smaller than 6 bytes (or in general, to compile a fixed maximum size into the kernel and to extract the actually used address bytes in the link level driver).

We used a different approach: as we noted that the routing table structure used in 4.4BSD, and thus in NetBSD, already supports arbitrary length addresses (and stores the length explicitly), we just used that information. (In 4.4BSD there is no seperate ARP table like in 4.3BSD and derived systems; rather, the ARP information is stored in the same radix tree structure as the network layer routing table. For a descripton of this, see [5].)The routing table resolves to routes, whose address information is stored in a struct sockaddr of the appropriate type containing a length byte.)

Section two points out essential data structures and some code sequences changed, while section three provides a walkthrough of porting an old-style BSD Ethernet driver to modern NetBSD.

2.  Some changed code

• We observed that the link-layer interface address is already in the interface address linked list (if_attach() puts it there). We added a field called if_sadl, pointing to that entry, for quick reference by drivers who need it.
• netinet/if_ether.h no longer exists. The really Ethernet-specific parts have moved to net/if_ether.h, the ARP specific parts are in netinet/if_arp.h.
• struct arpcom has been replaced by struct ethercom, after removing the link level address field. It only contains the link-level multicast-address list of the Ethernet drivers.
• struct ether_arp is no longer used. Instead, the variable fields in an ARP packet are addressed using the arp_sha(), arp_spa(), arp_tha() and arp_tpa() macros in net/if_arp.h. All of them take a single parameter, a pointer to the first octet of the ARP packet (without any link level header).
• The multicast IP to Ethernet mapping code was moved out of the ARP code fragments into the general Ethernet code in if_ethersubr.c.
• The ARP system to driver interface for output was changed: formerly, the ARP system would build all of the ARP packet, including the link level header. In the new code, an address type AF_ARP has been added, and the ARP system calls the interfaces' output function with a sockaddr containing only type (AF_ARP) and length (2). (The addressing information to use is contained in the ARP packet itself).
• I won't describe details of internal ARP code changes here. Other than using the changed data structures, the new code has to pass different-type parameters, sometimes. As we didn't have to change the IPv4 limitation, the would-be more ugly changes in the ARP code weren't necessary.
• For a description of what had to be changed in the average Ethernet driver, see the next section.
• Some split Ethernet drivers, with board-specific front ends calling board independent chip drivers, sometimes used struct arpcom to transfer the boards' Ethernet address to the chip driver. This had to be changed, by adding a parameter to the setup function called, to transport the Ethernet address.
• Some application programs had to be changed to use the new include files, but normally, they don't notice anything unless you want to add support for non-Ethernet protocols, to programs that actually note the change like tcpdump. A compatibility file netinet/if_ether.h is provided; it just pulls in the new include files and defines struct ether_arp, and creates a compilation error if it is used to compile a kernel. Kernel code should be ported correctly, and usually needs changes for other reasons already.

3.  Ethernet-Driver-convert HowTo⁺

If you want to port an Ethernet driver from an old-style BSD to NetBSD, you should read this section. If you want to write a driver for more than one BSD, one of them being NetBSD, you should do the same, just replace replace by put into the NetBSD branch of the #ifdef. Look at Matt Thomas' FDDI-Driver as an example.

Throughout this section, struct ifnet *ifp; is a pointer to the interface structure of "our" device and struct foo_softc *sc; is a pointer to our softc structure.

3.1  Include Files Needed

Instead of

#include <netinet/if_ether.h>

use

#include <net/if_dl.h> for the LLADDR macro,

#include <net/if_ether.h> for real Ethernet stuff,

#include <netinet/if_inarp.h> for IP-specific ARP stuff.

Sometimes you also need

#include <net/if_arp.h>

3.2  Softc Structure

• Remove struct arpcom from the beginning of the softc definition. In most drivers it is called struct arpcom sc_ac; or struct arpcom sc_arpcom; Throughout this text, we assume the former.
• Add struct ethercom instead. Throughout this text, we assume you called it struct ethercom sc_ec;
• In all references to the multicast list, use the struct ethercom, that is replace sc->sc_ac by sc->sc_ec.
• Replace all reference to sc_ac.ac_if by sc_ec.ec_if. [While you are here: I've seen lots of places, where, at the beginning of a function, a struct ifnet * parameter is converted to a struct foo_softc *, and later there are lots of sc->sc_if.bar or &sc->sc_if accesses. This is ugly, confusing and unnecessary. Depending on the nature of the driver you might want to change this while you're at it.]

3.3  Changed functions

• arp_ifinit() [4] takes a struct ifnet * as the first parameter, instead of a struct arpcom *. You may have noticed by now, that you don't have a hardware address any longer in the softc. There are two types of accesses to it. One is initialization, in the foo_attach() function (if this is done in the fooprobe() or foomatch() function, your driver is broken for other reasons and should be fixed); the other is runtime reads, and for crazy protocols like XNS, OSI, DECnet, even writes.
• Initialization: you call ether_ifattach() [3] with a 2nd parameter, a u_int8_t * pointing to the address. ether_ifattach() will create (as before) a struct sockaddr_dl in the interface address list, copy (among other initialization) the address there, and makes ifp->if_sadl point to it.
• Run-time: use LLADDR(ifp->if_sadl) as a caddr_t pointing to our Ethernet address. If you need the address length, you should probably use ETHER_ADDR_LEN. 

After you applied these changes, your driver should compile and run in the normal case. For more complex driver structures, some tweaking might be necessary as outlined at the end of the last section. Use the pre-existing code as a guide.

4.  Experiences and conclusions

4.1  Implementation

After outlining the needed changes, the actual implementation wasn't difficult, just lots of work. Most of this was to change the existing Ethernet drivers as described above. The merging of the changed tree back into the main sources using CVS took some time, but only one merging error was reported (and fixed by the messenger himself). Of course, before doing that, the code was thouroghly tested in a live environment (at the CS department of Bonn University).

A second (but small) part was to add ARP support to the ARCnet driver; this part of the code has been in use at the author's home network since then, and was heavy duty tested on a Ethernet-ARCnet router at an Amiga developer meeting.

The code has been merged into the main NetBSD tree early in the March of 1997, and has been available in the public developer tree [2]since then. It has been part of the NetBSD-1.3/1.3.1/1.3.2 formal release.

4.2  Performance

No performance loss was actually observed. This was expected for two reasons:

1. The most heavy per-packet penalty would come from the use of variable field length macros instead of fixed structure offsets in the actual handling of ARP packets. However, this only affects ARP packets (a total of a couple of machine instructions per ARP packet); once ARP has resolved an address, it is cached in the routing table and applied (when sending packets) like before, with a timeout of several seconds.
2. An additional penalty comes from inserting the sender Ethernet address into the packet header. Here, instead of getting the address from a fixed offset in the ifnet structure, an additional indirect address fetch and offset calculation are needed. This might be noticeable if you do sub-microsecond profiling of the kernel networking code, but is negligible compared to the other parts of the transmission path.

4.3  What do I see of this code?

The normal user won't notice it at all, if handling Ethernet. You might, however, notice ARCnet support with ARP on NetBSD/Amiga (and on NetBSD machines with ISA busses, soon, I promise!), or support for AX25 packet radio, once a driver is available, which was made possible by these changes.

References

[1]

Plummer, D., An Ethernet Address Resolution Protocol, RFC-826, Network Information Center, SRI, November 1982

[2]

The NetBSD public developer version of the source code is availabe through FTP from ftp.netbsd.org, subdirectory /pub/NetBSD/NetBSD-current/

[3]

ethersubr(9): common Ethernet driver-independent functions documentation in the NetBSD Kernel manual

[4]

arp(9): ARP subsystem man page in the NetBSD Kernel manual

[5]

Wright, G.R., Stevens, W.R.: TCP/IP Illustrated, Volume 2 (The Implementation), Addison-Wesley, 1995

Footnotes: