1 Background

I have been using FreeBSD for nearly a decade, and I continue to be impressed by its overall reliability, availability, maintainability, and applicability. In my former occupation as a part-time system administrator and part-time college instructor, I relied on FreeBSD for infrastructure services like Web, DNS, and FTP servers even though our site was predominantly Sun workstations. In my current job as full-time college professor, I have less and less time to spend with system administration tasks, but more and more demands on my time to keep classroom activities current and interesting for the students.

A faculty member decided to offer a course in graphical game programming. My current site is also predominantly Sun workstations, but the entry-level graphics cards on the Suns were not up to the task of some of the demanding graphics requirements such as texture-mapping. Since we were determined to offer the course, we discussed the possibility of buying the high-end graphics cards for our workstations, but instead chose to spend about the same amount of money (actually, a bit less) on five new PC’s with game-quality graphics cards. We needed to obtain and deploy a small number of stations without exhausting our budget or straining our already overworked system administrator.

2 FreeBSD to the Rescue

Once we decided to buy the PC’s, FreeBSD became an obvious part of the solution. With my experience, I knew FreeBSD would not require any special hardware, and that a wealth of software was available via packages and ports to quickly set up a small cluster of machines. Furthermore, I knew that once the machines were set up, they would require little or no maintenance over the course of the semester. Finally, I was confident that we could reuse the machines in the future, for possibly a completely different goal by simply upgrading and changing the application mix. As a “bonus”, the availability of PC’s also meant we could dual-boot Windows and FreeBSD and expect the students to also grasp cross-platform development troubles and techniques.

I was able to buy five computers each with 3.0 GHZ HTT Pentium-4, 512MB RAM, 80MB hard drives, DVD, and 256MB graphic cards in a mid-tower case. The total cost was under $3000, which was reasonable at that time. We had plenty of monitors, keyboards, and mice in storage.

In addition, I made use of an older system as a server for this cluster. This server would be providing some file and NIS (yp) services, so it did not require any cutting edge hardware. In addition, we would be able to use the existing Sun servers systems for the student’s home directories, which meant issues like user file backup were not my problem.

1

3 Step-By-Step

The first step was to assemble the PC’s and install the operating systems. Luckily, we have a computer club, specifically a student branch of the ACM, and several students were only too happy to come in over the course of two nights to do these important but sometimes tedious tasks.

In parallel to the above, I would be installing the server. FreeBSD 5.3 had been released and by all reports was stable enough for our applications. I followed my usual installation routine of first installing from CD-ROM, then updating the system sources to the most recent STABLE release from the Internet and bringing the operating system up-to-date, and then compile a custom kernel.

Once the server was at the latest release, I started to configure the important services like NIS and NFS. The only NIS setup that was necessary was to make the machine a slave server in our existing NIS domain. For NFS, I decided that I would export /usr/local and /usr/X11R6 from the server. Most application software installs to /usr/local, so that was an obvious choice. Some X applications also install in /usr/local, but many install in /usr/X11R6 so I choose to export /usr/X11R6 as well. Since this course was graphics intensive, I wanted to have the option to update or modify X11 for the entire cluster as quickly as possible if it became necessary. I considered mirroring, but I did not think that would add any functionality, and it would require the time to perform the mirroring of the 1.7GB in /usr/local and then 900MB for /usr/X11R6 that would be present in the final system.

I did encounter one “gotcha” with exporting /usr/X11R6. The directory /usr/X11R6/lib/X11/xkb/compiled needs to be writable since xkb writes it compiled keyboard definitions there on what appears to be a per-session basis. I did not want the server exported directories to be writable by the clients, so I had to make this directory a symbolic link to /var/tmp. With this small customization, each client now had its own local compiled keyboard configurations.

Only two “small” steps remained.

1. Install the software that would be needed by the students.

2. Configure the client machines to connect to the server.

Since some of the packages to be compiled were quite large and would require a fair amount of time to compile, I was able to parallelize these two steps to some extent. While some software was compiling, I would create and test small “assimilation” scripts to be used to configure the clients.

4 Summary

After each of the clients were assimilated via my configuration scripts and all the software was installed on the server, the cluster was ready. If there had been no interruptions, the entire process could have taken a few days, but it took about a week of real time. Once the systems were up and running, the only “maintenance” I performed on them was to create custom kernels on the clients as time permitted. The approaches and tools used for the lab worked out incredibly well. By leveraging tools like portupgrade and the Ports system with some good system administration skills, it was possible to create a small special purpose cluster of workstations within the context of the rest of my workload. In addition, making use of the same tools can be used to keep them up-to-date and reconfigure them for other special purposes as necessary.

3