Daemon News '200312' : '"Cross-building Explained: Cross compilation on NetBSD "'

DæmonNews: News and views for the BSD community

Cross-building Explained: Cross compilation on NetBSD

by Erik Berls

Introduction

You've just unpacked that Dreamcast you got off eBay, have it booting, and now you want to put a tuned kernel on it. However, you'd rather not spend hours waiting for it to build. Especially since you've got that P4 or Athlon XP system sitting on your desk.

The solution? Cross building.

Starting with NetBSD 1.6, the build infrastructure radically changed for the better.

The crossbuild framework allows a host to build a version of NetBSD 1.6 or later regardless of the version of the host. This framework allows the full release sets to be built in this manner, without needing to be root, or have any other tools on the host except a compiler. To make things easier, the crossbuild support in NetBSD is built directly into the standard build framework.

General benefits

Crossbuilding has several major benefits depending on your environment. If you have production servers, you can build the OS without needing to load down the machine that is actively surviving as a production host. Smaller, special purpose boxes can easily be kept up to the latest on their branch.

Tracking the development branch is greatly simplified. One no longer has to install the latest version of the compiler native to the system in order to build -current.

One may wish to rebuild the tree with a specific set of compiler flags, or just the latest on a branch. The build system will produce installable sets that can be used to upgrade other machines without the need for compilers or source trees on the target machine. In addition, these installable sets can be used as a backup.

Why did we do it?

As NetBSD has grown, the amount of co-ordination needed to produce a release was drastically increasing. The 1.5 release consisted of 20 systems (ports). The 1.6 branch contains 40. NetBSD 2.0 will contain even more.[1] In order to produce a release, Release Candidates (RCs) had to be produced for people to test. This used to take at least a week from start to finish on some of the slower architectures, assuming the developer who owned the hardware could start right away. With the introduction of the crossbuild framework, this process was reduced from an impractical 10-11 days, down to just under 28 hours.[2]

NetBSD is frequently used in embedded systems.[3] While some of these systems are definitely slower than most modern desktop machines it is the lack of a good environment that makes it difficult to develop directly on the specific piece of hardware. The ability to build the required binaries on a separate host greatly reduces development time. Coupled with the ability to develop the application specific code on NetBSD based desktop and server machines, extensive and sophisticated development environments can be used, and expensive embedded systems are not needed for every developer.

Many people prefer to keep compilers off of production servers. These machines may not always run the latest major release of the operating system, but instead an older release that has been well tested. With NetBSD 1.6, and moving forward, security fixes and other changes can be rapidly build without requiring that a dedicated host be kept around. Wasabi Systems, Inc. funded much of the work on this system. They utilize NetBSD code in embedded systems and have contributed back to benefit the NetBSD community as a whole.

How did we do it?

The cross compiler is only one element, albeit a very import one, of the entire process. The key was to determine what the host specific elements were in the build process.

By determining what specialized code was run and its dependencies the framework was adjusted to allow not only cross building from other NetBSD hosts, but other POSIX compliant hosts as well!

This means that the host system doesn't even need to be a NetBSD box in order to build NetBSD! In fact people have built NetBSD on FreeBSD, Linux, and even Windows machines that were loaded with Cygwin.

Starting builds

Since the framework is to be used as the entry point to building the source tree it makes sense for the framework to be as simple as possible.

Only a few flags are necessary to start a build. In fact, merely typing ./build.sh will start a build for the hosted architecture if run on a NetBSD host, with a selection of default directories.

The first thing that happens is that the framework uses the host tools to build a set of cross tools, starting with make. In the first stages a cross-compiler is built. This ensures that NetBSD itself is built with a compiler that is known to be the correct version. The tree is then built.

A key flag that can be utilized is '-U' to allow a non-root account to build the full distribution. A special mechanism is used to preserve the file permissions and create the .tar files with the correct permissions. Flags can be used to direct the installation of the binaries and build distribution sets. Additionally, flags can be set to direct where various object files, destination files, and release files are stored.

To allow the building of a different port than the machine hosting the build the '-m' flag is used. This flag specifies the target port for the build mechanism, and directs the creation of the appropriate cross- compiler. This flag is used extensively in the AutoBuild framework.[3]

Cross building other software

"Wow! This is really neat! I want to build *all* the software for my slower architectures this way!"

Whoa! Not so fast. A lot of work went into getting the NetBSD source tree to the point where this could all happen. Certainly, certain pieces of software can be crossbuilt, but the more complex software attempts to figure out elements of its environment and adjust itself to match. This causes problems.

Even very simply configured software can be a pain to cross build. Many scripts, such as autoconf, embed the specific executable name for the compiler into Makefiles, etc. A good deal of work would need to go into making autoconf and friends crossbuild aware.

The other challenge is to deal with software that builds small test programs and runs them making decisions based on the output, or in the creation of data files. In NetBSD 1.6 file(1) exhibited a failure mode when it was cross compiled from a LP32 to a LP64 machine. This was fixed in a later version of file(1).

Next Steps

Currently, xsrc is being reshaped to support crossbuilding for NetBSD 2.0. Other features have been added to the ./build.sh script in NetBSD 2.0 that make things easier for the end user.

As we move closer to the entire system being crossbuilt, we can automate much of the final steps in many development and release processes.

Part 2 of this article series will go into additional details of the options, how to effectively utilize them and the additional features in NetBSD 2.0's build.sh.