A Tour through the NetBSD Source Tree: Part II - Libraries

Hubert Feyrer <hubert@feyrer.de>

In Unix(-like operating systems), commonly used routines that can be accessed from many programs are grouped within so-called libraries and that can be used from application programs. The src/lib directory contains the libraries that come with NetBSD, and there's quite a number of them. Let's have a look!

csu:

Sources for crt0.o and it's C++ cousin c++rt0.o. These are not libraries, but equally important nevertheless, as they do the startup of every program - pull in shared libs, for C++ call any constructors, then call main(). When main() returns, several cleanup functions are called that call e.g. C++ destructors. After that, the return code from main() is returned to the calling program and the program terminates. Sources in this directory are partly coded in machine language for efficiency, and there are special functions for each architecture and object file format (a.out, ELF).

libarch:

This library contains some architecture-specific functions that might be useful to access from userland programs. For i386, functions exist to manipulate the per-process I/O permission bitmap, I/O privilege levels and set virtual 8086 processor registers and mode. Other archs have functions for handling CPU caches from userland here.

libbz2:

The bz2 algorithm compresses files using the Burrows-Wheeler block-sorting text compression algorithm, and Huffman coding. This library is used in several programs to allow reading and writing of bz2 compressed archives and files.

libc:

This is probably the most important library in NetBSD, as it contains the "trampoline" stub code to do kernel calls from C programs. The stubs are tiny pieces of C/assembler code that take any arguments for system calls (check your section 2 of the NetBSD manual pages, starting with intro(2)!), then bring them into shape to do a system call using whatever way the underlying hardware uses to do so, usally a trap or software interupt. The system call is executed in kernel context, and any return value is taken by the stub and fed back too the calling routine. Besides the stub code for system calls, libc also contains code for a lot of convenience functions that can be called from programs without linking in any special library, as libc is always pulled in automatically. These convenience functions include code for

• database access using the Berkeley DB code
• support functions for shared libraries, which do load shared objects (dlopen(3), ...)
• assisting profiling
• various hash routines (sha1(3), md5(3), ...)
• locale and NLS handling
• processing network related information, like the name service switch routines and the DNS resolver code
• regular expression handling
• RPC handling including service creating/accessing for both clients and servers and XDR handling
• routines for 64bit type handling, for 32bit CPUs
• software floating point operation routines for machines that don't have a floating point unit (FPU)
• the "stdio" set of functions that handles buffered input/output and many other things.
• the "stlib" functions
• string manipulation functions
• the terminal I/O subsystem that allows hardware independent programming of terminal driven software, by using a translation table for terminal functions, termcap.
• timezone-handling functions
• network information service (NIS) handling

Also important, this directory contains all the manpages for the system and library calls (manual pages section 2 and 3).

libasn1, libcom_err, libhdb, libkadm, libkadm5clnt, libkadm5srv, libkafs, libkdb, libkrb, libkrb5, libkstream, libroken, libsl, libss, libvers:

These directories contain various auxilliary libraries for both Heimdal and MIT Kerberos. Unfortunately, there is little documentation about the functions and interfaces they provide, and they are currently used by Kerberos only.

libcompat:

The routines collected here are for backward compatibility with old software interfaces, and must be linked in explicitly. Included here are the regexp(3) routines which were obsoleted by regex(3), and various functions for terminal handling, queue management and remote program execution.

libcrypt:

This library contains (only) the DES routine used for password encryption. It was put into it's own library so it can be omited easily from systems that are exported to countries which fall under the US export restrictions. Fortunately these are much less strict nowadays than when libcrypt was invented.

libcrypto:

This library contains various routines for data encryption, decryption, hashing and signing. The functions are part of the OpenSSL distribution, and the source is accessed using the reachover mechanism, the actual sources are in src/crypto.

libcrypto_idea, libcrypto_rc5:

While export restriction and movingg restricted functions into their own libraries, taking care of existing patents is another issue. The IDEA and RSA RC5 algorithms that come with OpenSSL are patented, and may not be used under certain conditions. To allow disabling them easily, they were moved into their own libraries again. Using shared library techniques, libcrypto will pull them in if available, and fail with an abort(3) call if they cannot be found (when called).

libcurses:

This directory contains sources to the curses(3) terminal handling library, which can be used to do simple, terminal/screen-based applications in a hardware/terminal independent way. Note that NetBSD does not use the ncurses implementation, but one that is derived from 4.4BSD and that was extended to follow the latest standards.

libdes:

The libdes that existed on 4.4BSD systems and that provided routines for the Data Encryption Standard for encrypting and decrypting data are contained in the OpenSSL-based libcrypto these days, and libdes is just a dummy directory that keeps information about the libdes shared library. The library itself is a link to libcrypto and is generated when libcrypto is built/installed.

libedit:

This library contains a set of functions for command line editing and history processing. It originated in 4.4BSD and was extended to provide source-level compatibility with GNU's readline library by the NetBSD project.

libgssapi:

This library implements the Generic Security Services, which provides security services to callers in a generic, source-level compatible way, and which usually sits above the cryptographic libraries. See RFCs 1508 and 1509 for more information, there's (unfortunately) not much documentation contained in this library, which is mostly used by Kerberos.

libipsec:

These routines convert IPsec policy specification structures from and to (human-readable) string, and allow accessing the pfkey API.

libkvm:

The kvm(3) library provides a uniform interface for accessing kernel virtual memory images, including live systems and crashdumps. Available functions include retrieving the current load average, a list of open files as well as routines to access arbitrary symbols for both reading and writing. This library is used by many programs to retrieve status information from various kernel subsystems.

libl:

The (f)lex lexical analyzer has some common routines that are stored in this library. The sources are taken directly from src/usr.bin/lex by reachover Makefiles. The resulting library is installed as both libl and libfl, as the "lex" that's shipped with NetBSD is really the "flex" implementation.

libm:

The math library contains many functions for single and double precision floating point arithmetics, trigonometric functions and many more. This library must be linked in explicitly when using simple math functions like sin(3) and cos(3).

Many of the functions in this library exist in specially optimized versions written in machine language for various CPU/FPU architectures, for maximum performance. There is also the distinction between machines using IEEE floating point format internally and these that use a different format, which is reflected in the source.

libmenu:

The menu(3) provides a terminal independent menu system using the curses(3) library.

libossaudio:

This library provides compatibility to the Open Sound System API. It can be linked against sources that (usually) originate on Linux, and the OSS sound calls will be mapped to use the native NetBSD sound system. Beware that for using this library, the needed header file <soundcard.h>'s path is different than on Linux (which uses <sys/soundcard.h>)

libpcap:

The Packet Capture library can be used to implement packet sniffers and other applications that need to know about network traffic in a machine independent, efficient way. It's used e.g. by tcpdump(8).

libposix:

This library has code for some system calls that by default implement some non-POSIX API, e.g. the traditional 4.4BSD APIs. To get calls with real POSIX semantics, libposix can be used. The code for this library is taken from libc (using reachover Makefiles), POSIX semantics are enabled using compiler switches at build time. Available functions here include chown(2), lchown(2), fchown(2) as well as rename(2). See the "STANDARDS" sections of the corresponding manual pages for more information.

libresolv:

This library implements special functions for talking to the DNS resolver. While general resolving functions are built into libc, functions here can be used to tune behaviour of the resolver, hand-craft DNS queries and several other things. See resolver(3) for description of the functions available here.

librmt:

If you need operations on remote tapes, this is what you need. This library is used by dump/restore and other applications to use remote tape drives. See the rmtops(3) manpage for more information.

librpcsvc:

Several commonly used RPC server handlers, for a number or services and protocols: bootparam, NFS, mountd, rquota, rstat, NIS, etc.; The source exists in the form of .x files, which are used as an input to rpcgen(1) to produce interface definition (.h files) and server stub code (.c files), the latter of which is then put into librpcsvr.

libskey:

The s/key one-time password library is used by several programs to implement disposable one-time passwords that can be used to authenticate from insecure environments. See skey(1) on how to setup s/key, which can then be used when logging in via FTP, telnet and other services.

libssl:

The secure sockets layer (SSL) library is compiled from the OpenSSL sources located in src/crypto/dist (see below) using reachover Makefiles.

libtelnet:

This library contains various auxilliary routines used by telnetd(8), e.g. functions to utilize Kerberos authentication.

libterm:

This is the directory that contains sources for libtermcap, which implements hardware-independent operations for accessing terminal devices. Routines are provided to lookup hardware-independent operatins, and provide terminal-dependent operations using the termcap(5) terminal capabilities database.

libusb:

The usb library provides routines to extract data from USB Human Interface Devices, e.g. identification of mice and keyboards, keyboard mapping, number of mouse buttons etc. Furthermore, data on certain events like key presses can be extracted.

libutil:

This library contains various useful routines that are used in many utilities that come with NetBSD. Routines include line parsing, finding out about the number of maximum partitions supported by the port the program's running on as well as determining the number of the raw partition ('c' vs. 'd'), login capabilities, as well as terminal, pseudo-terminal, disk and password file handling.

libwrap:

The TCP wrapper library is used to authenticate peers before accepting network connections from them. It's used by several services in NetBSD, with inetd only being the most important one - every service started via /etc/inetd.conf can be authenticated without adding code to the individual services. Other services not started by inetd that can still make use of the TCP wrapper mechanism include rpcbind, sshd, supfilesrvr and ypserv. Documentation for adding similar authentication for application programs is available in the hosts_access(3) manpage.

liby:

This library is only used by the yacc(1) compiler generator, defining some auxiliary functions that are not intended for use by random 3rd party applications. Parsers generated with yacc(1) will need to be linked against this library, though.

libz:

The libz compression library is used by gzip and various other applications to read/write compressed data, e.g. install-info, ssh and makewhatis as well as a lot of third party applications from pkgsrc. The sources of the library are available here, unfortunately the documentation consists of source and header files only. UTSL! :-)

As you can see, there is quite a number of libraries available to be used by applications and programmers. In addition with the source layout for programs described in the first part of this series, this is all that makes up the NetBSD operating system's userland. In the next part, we will have a look at the part that's not user-visible, diving into the kernel sources.