DæmonNews: News and views for the BSD community

Daemon News Ezine BSD News BSD Mall BSD Support Forum BSD Advocacy BSD Updates

Trawling the Ports Collection

Greg Lehey <grog@lemis.com>

One of the most important parts of the BSD distributions is the Ports Collection, called the Packages Collection in NetBSD. It neatly solves the problem of how to get third party software running on your machine, and there's something in there for everybody. The idea came from FreeBSD, which still has the largest collection of ports--nearly 7,000 of them, in fact, including just about anything you can think of. A large number of programs that are part of Linux base distributions are in the BSD Ports Collections.

Of course, you can have problems with the Ports Collection:

  • How do you know what's there?

  • We continually hear complaints from converts from Linux: ``I hate this csh shell. Where's bash?''. It's there of course--in fact, there are two different versions--but where?

  • How do you know how to use the ports? Often the documentation is less than perfect, or you don't know where to find it.

Let's consider finding bash. If you're running FreeBSD, you could start with link to the index page on the FreeBSD home page, where you'll find a list of the ports categories and links to more details. But what category is bash in? There are 64 categories, and you may not know where to look for bash. By the time you get to Shells, somewhere near the bottom, you've compared so many others that may no longer be sure. And that's just for a program you know and you're looking for. How can you find things you don't know? And once you've found it, how do you use it?

This is the first of a series of articles which addresses these issues. In particular, every month I'll dig out an interesting port or two, but first, this month I'll give a brief introduction to the Ports Collection and the environment in which I do my work. Hopefully you'll find it interesting; one way or the other, I'd be interested in feedback.

Installing ports

I'm assuming here that you know the basics of the Ports collection. If not, read James Howard's article in the June 2000 edition of Daemon News. That article mentions the standard way of installing a port, using make. In fact, you have three alternatives:

  1. You can build the package from source, as described in the article. This is the standard way, and it is the most flexible, but it is also the slowest.

  2. You can get a pre-compiled package and install it using pkg_add. A package is a tar archive which contains the installable results of building a package. You just need to install it, so it is a lot faster. On the down side, there may be version dependencies, such as on libraries, which mean that the installed result may just plain not run, or it may create spurious error messages. For example, while writing this article, I was given an OpenBSD package for xtset, which we'll look at below. It was built on a -CURRENT OpenBSD machine, and I'm running 3.0. This is what I got:
    $ cd /usr
    /usr/libexec/ld.so: warning: libc.so.28.0: minor version >= 3 expected, using it anyway
    /usr/libexec/ld.so: warning: libc.so.28.0: minor version >= 3 expected, using it anyway
    $
    
    The error messages come from xtset. In this case, they're benign, but similar problems could stop the package working altogether.

  3. If you don't have the package locally, you may still be able to install a remote package. The way you do this depends on the system you're using: NetBSD and OpenBSD require an explicit URL, while FreeBSD can work out the URL given the package name. For example, on FreeBSD you could install zsh with the command:
    # pkg_add -r zsh
    Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-stable/Latest/zsh.tgz... Done.
    updating /etc/shells
    
    With NetBSD and OpenBSD, you need to find out the location first. Then you write something like:
    # pkg_add ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.5/i386/shells/zsh-3.0.8.tgz
    ftp: connect to address 3ffe:8050:201:1860:2a0:c9ff:feed:b7ea: No route to host
    setenv PKG_PATH='ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.5/i386/shells'
    missing: ./libexec/cgi-bin (created)
    missing: ./share/mk (created)
    missing: ./share/nls/uk_UA.ISO_8859-1 (created)
    
    That ``No route to host'' message is benign: ftp.NetBSD.org has an IPv6 address, and ftp tries it first. My system isn't connected to an IPv6 network, so there's not much hope of it working. The real issue here, though, is knowing that the zsh you're looking for is at ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.5/i386/shells/zsh-3.0.8.tgz. I normally find these things out by searching the ftp site. Once you're at the site, of course, it's easier just to fetch the file locally.

What's in a port?

But what is zsh? Why should you want to install it? Those are the questions that I'm going to be addressing in these articles. If you look carefully at the example above, it looks as if it should be a shell. But what kind of shell? Well, there's the documentation, of course, but it would take a lot of time to read the documentation for that many ports.

The Ports Collection acknowledges this problem and includes summary documentation. Again, the location depends on the version of BSD you're running:

  1. FreeBSD supplies two files: a one-line description in pkg-comment and a multi-line version in pkg-descr. These files are in the base directory of the port. For the zsh we're looking at, they have the following contents:

    pkg-comment
    The Z shell
    
    pkg-descr
    Zsh is a UNIX command interpreter (shell) which of the standard shells
    most resembles the Korn shell (ksh), although it is not completely
    compatible. It includes enhancements of many types, notably in the
    command-line editor, options for customising its behaviour, filename
    globbing, features to make C-shell (csh) users feel more at home and
    extra features drawn from tcsh (another `custom' shell).
    
    If you want to use zsh completion system, you should type the following
    commands:
    
    	$ autoload -U compinstall
    	$ compinstall
    
    See also zshcompsys(1) manpage. :)
    
    WWW: http://www.zsh.org/
    
  2. In NetBSD and OpenBSD, there's only one file, pkg/DESCR, which in general correspond to FreeBSD's pkg-descr. In this case, they contain the first paragraph of the pkg-descr above.
So, are you happy with that description? Neither am I. There's more documentation with the package, of course, but nothing telling you ``use zsh because of its really neat command line completion'', or maybe ``I don't use zsh because it's such a pain to set up the command line completion to do exactly what I want to do''. Which is correct? That's a matter of opinion. These articles are my opinion.

Getting started

So, what shall we start with? I have a few favourites which I install on every system I use for any length of time. We'll look at the shell environment this month.

A shell: bash

I've already mentioned bash, the Bourne Again Shell from the Free Software Foundation. It's one of a plethora of shells available, and it's the one I use. Why do I use this one? I have a number of reasons:

  1. I've been using it for about 13 years, more or less since it first became available. Before changing, I need a reason.

  2. It has nice command line editing, better than most. tcsh has very similar editing, but either it can't do some of the functions that bash can, or I haven't found out how. See #1.

  3. It supports Bourne Shell syntax. This is very important: you need to know Bourne Shell syntax for any serious shell script. Hardly anybody writes scripts for csh or derivatives, so why learn two shell syntaxes?

Alternatives

There are a number of good alternatives to bash, of course:

  • I've already mentioned tcsh. The command line editing is quite good, but it doesn't seem to understand the Meta (Alt) key. It also has csh syntax, which invalidates it for me.

  • I've also mentioned zsh. I tried it for a while, but I never got the command line completion configured the way I want it. Maybe that's been improved in the meantime, but I have better things to do with my time.

  • There are a number of versions of the Korn Shell, ksh. OpenBSD ships with one which seems quite good. But again it doesn't seem to have as good command line editing as bash.

Installing bash

We've seen how to install ports. Simply do:

# cd /usr/ports/shells/bash2 (FreeBSD, OpenBSD)
# cd /usr/pkgsrc/shells/bash2 (NetBSD)
# make install
That's straightforward enough. Of course, after installation you're still stuck with the same old shell you had before. You need to run chsh. This will give you an editor with details of your account, which looks something like this:
#Changing user database information for grog.
Shell: /bin/sh
Full Name: Greg 'groggy' Lehey
Office Location: Echunga SA
Office Phone: +61-8-8388-8286
Home Phone: +61-8-8388-8250
Other information: 
Just change the /bin/sh on the second line to /usr/local/bin/bash, save and exit, and you're in business. Well, sort of. Next time you log in, you should have a bash shell. But how do you use it? Where's the documentation?

The documentation

bash comes with a lot of documentation, including a print manual and two man pages (bash(1) and bashbug(1)). The same content is also available as info pages. info is the GNU documentation system; we might look at it in a later article. At the moment you can live without it.

bash(1) currently runs to 5144 lines, nearly 100 pages. It's probably not the sort of thing you'd want to read on line, but it's handy for searching. By contrast, bashbug(1) is very short and just tells you how to report a bug. The real documentation is in the 138 page reference manual. But where does it go? It's in /usr/local/share/doc/bash. It's available in PostScript and HTML formats:

$ ls -l share/doc/bash/
total 2
-r--r--r--  1 root  bin   86238 Mar 27 14:28 article.ps
-r--r--r--  1 root  bin   48004 Mar 27 14:28 article.txt
-r--r--r--  1 root  bin  243837 Mar 27 14:28 bash.html
-r--r--r--  1 root  bin  408887 Mar 27 14:28 bash.ps
-r--r--r--  1 root  bin    8043 Mar 27 14:28 bashbug.ps
-r--r--r--  1 root  bin  385523 Mar 27 14:28 bashref.html
-r--r--r--  1 root  bin  654109 Mar 27 14:28 bashref.ps
-r--r--r--  1 root  bin  125347 Mar 27 14:28 builtins.ps
You still need to print it out, of course. That's straightforward enough if you have a PostScript printer, but otherwise you'll need ghostscript, which we'll look at some other time.

Some interesting customizations

Shells are infinitely customizable, and since they're one of the two main tools you use to interface with the computer (the other is the editor), it's worth making things comfortable.

Everybody has his own views on this, of course. Here are some of mine. I put this in my .bashrc file:

case `tty` in
  /dev/ttyp*)
    TERM=xterm-color ;;
  /dev/pts/*)
    TERM=xterm-color ;;
  /dev/ttyq*)
    TERM=xterm-color ;;
  /dev/ttyv*)
    TERM=cons25 ;;
  /dev/ttyE*)
    TERM=cons25 ;;
  /dev/ttyC*)
    TERM=vt220 ;;
  *)
    TERM="" ;;
esac
export TERM
These definitions set the TERM environment variable depending on the device name.
if [ "$TERM" = "xterm" -o "$TERM" = "xterm-color" ] ; then
  PS1="\[^[[34m\]=== \u@\h (`tty`) \[^[[31m\]\w\[^[[34m\] \# ->\[^[[30m\]\[^[[47m\] "
else
  PS1="=== \u@\h (`tty`) \w \# -> "
fi
PS2="\u@\h \w \! ++ "
The sequence ^[ is in fact an escape character. These definitions first check what kind of terminal I'm running on. If it's an xterm, it sets a prompt in colour, which makes it easy to distinguish on a large screen:
=== root@kimchi (/dev/ttyp1) /etc 20 -> 
If it's not an xterm, the prompt is monochrome, but otherwise the same. This information includes:
  • The first part, ===, is just to make it easier to find in a large list on an X display, in the same way that the colour helps. An xterm window (a terminal emulator which runs under X11) on a high resolution X display can contain over 100 lines, and searching for command prompts can be non-trivial.

  • Next, root@kimchi is the user ID and the name of system on which I am working, in the RFC 822 format used for mail IDs. Multiple systems and multiple users can all be present on a single X display. This way, I can recognize which user I am and what system I am running on.

  • /dev/ttyp1 is the name of the current controlling terminal. This can sometimes be useful.

  • /etc is the name of the current working directory. When you're in your home directory, it is represented as ~ (tilde). Most shells, but not all of them, support this symbolism.

  • 20 is the prompt number. Each time you enter a command, it is associated with this number, and the prompt number is incremented. One way to re-execute the command is to enter !!20 (two exclamation marks and the number of the command), but bash offers better solutions.

Next, a function definition:

Make ()
{
  if [ "${MAKE}" = "" ]; then
    MAKE=make
  fi
  xtset `uname -n|sed 's:\..*$::'`:`pwd`: Make "$* "`date`
  (echo '=====' `date`: Make $*; /usr/bin/time -l ${MAKE} 2>&1 $*) | tee -a Make.log
  cd .
  }
This defines a function called Make. As the name suggests, it's like make. It runs make and copies the output to a file called Make.log. If something goes wrong in a make, you can then go back and look at Make.log to see what happened. It also uses xtset to set the title of the xterm to information about the build. For example, when building a FreeBSD kernel, it is set to:
zaphod:/usr/src/sys/i386/compile/ZAPHOD: Make all -j2 -k Wed Mar 27 18:52:47 CST 2002
Finally, it times the build, so I can have more of an overview over what's going on.

What, you don't find xtset on your system? No, that's not surprising. It's in the Ports Collection. Well, in the FreeBSD Ports Collection, anyway.

xtset

By comparison, xtset is tiny. It takes an argument which sets the title of an xterm. If you're not running on an xterm, it does nothing. There's really not much else to say about it, though of course it comes with a man page (xtsert(1)). Here's another shell function which takes advantage of it:

cd ()
{
  command cd $*
  if [ $? = 0 ]; then
    xtset -t "%u (%T) %h:%D"
    xtset -i "%u@%h:%D"
    true
  else
    false
  fi
}
This command redefines the cd builtin command. First it executes the builtin, then it uses xtset to set the titles separately for the xterm window and the icon. The format is similar to that of the prompt. When in my home directory, the title is
grog (ttyp1) wantadilla:~
This can get rather long for an icon, so the icon format is different:
grog@wantadilla:~
xtset is in /usr/ports/x11/xtset on FreeBSD. If you're using NetBSD or OpenBSD, don't despair: pick up the tarball from ftp://ftp.lemis.com/pub/xtset-1.0.tar.gz and install it manually. It's not quite as easy as installing it from the Ports Collection, but it's not that difficult either.

But why should you bother? There are good reasons: as I write this article, I have 77 windows open, spread across 5 screens. More than half of those are xterms, spread over about 10 machines. Just finding which is which becomes a challenge; xtset helps.

More information about the Ports and Packages collections

These articles aren't intended to be a complete guide to the Ports Collections. If you don't have any background in installing ports, you may find it heavy going. You can get more information from these links:

Google
Web daemonnews.org

More Articles
  • Interview with Jan Schaumann
  • Interview with Theo de Raadt
  • Book Review: Virtualization with VMware ESX Server
  • Editorial: Not Quite Dead Yet
  • The Design of OpenBGPd
  • Interview with der Mouse
  • Letter to Steve Jobs
  • Interview with Manuel Bouyer on Xen
  • Apple and Open Source
  • BSDCan 2006
  • BSD Certification Survey Results
  • Lab in a Box
  • Ike Notes on BSDCan 2005
  • BSDCan 2005 Photos
  • FreeBSD Developer Summit Pictures

  • Advertisements




    Author maintains all copyrights on this article.
    Images and layout Copyright © 1998-2006 Dæmon News. All Rights Reserved.