Answerman: Shell history overview, part 1 of 2

The AnswerCrew

This time we decided to write something on shells avilable on every Unix and Unix-like operating system. There are many articles and books on that topic but we want to show our own point of view, share the experience, configs and describe problems you may face.

From simple introduction on shlls we will move on to initialization files for every shell we know, environment description, syntax. We will show you how to work with files and directories. Looking for something and don't know how to find it? Do you have a problem with file permissions, basic process and job control? Fine you will learn from this, the first of a two-part article. At the end we talk on programming basics, and a little on security.

We hope you will find it useful.

For all the AnswerCrew,
    Grzegorz Czapliñski

In this Brief Historical Overview of the Unix Shell, let's begin with the definition of "the shell". Today, since shells are part of the command line interface (in contrast to the graphic user interface that is more widely used) most computer users have not the faintest idea of what a shell is or does. Briefly put, a shell is an interpreter of the commands that the users type. The shell is the interface to the kernel, the operating system. Modern shells understand such miscellany as pipes ("|") and the logical AND's and OR's ("&&" and "||"), do shell redirection using ("<"), (">") and (">>"), and the symbols used to run a command in the background, the ampersand, variable substitution, and lots more. The shell also keeps track, internally, of the error status given it by the utilities or other programs it invokes so that you can automate decision in a script or on the command line.

So if you type at the command line

	$ cat fooA fooB fooC | tee fooTotal |egrep -w "the|their" && echo YES.

the shell knows to concatenate the first three files into a larger file, look for one of two explicit patterns and if egrep found either pattern, the shell will print "YES" to the screen.

The advantage to knowing command-line interfacing is pretty obvious if you have ever tried to do anything complicated through a GUI. If you work mostly with the keyboard, you are saved mousing around; the more you learn, the faster you can learn, and with some thought, you can efficiently accomplish myriad administrative tasks.

A few words on the most popular shells in Unix world

Steve Bourne, at Bell Labs, wrote /bin/sh in 1974; it was the first shell ever written and it helped Unix stay with the its Prime Directive: use many small, flexible programs as tools to get any larger job done. /bin/sh, a.k.a. sh, lacked the ability to make interactive use such as a history log.

Although the Bourne shell was the first and very simple, it became the model for many following shells; also, later versions of /bin/sh were much improved. The Bourne shell did not have a command history, job control, command line completion or aliasing capability. It did, however, have the ability to use shell "functions" that were defined in the shell's initialization file, ~/.profile.

The shell /bin/sh in the BSD distributions is not Steve Bourne's original shell, but "ash" (A SHell) written by Kenneth Almquist in 1989 and has been improved by various people since. ash, now renamed sh, is a superset of the original Bourne shell.

In 1978, at Berkeley, Bill Joy created his C-shell with several innovations among them "built-ins" where several of the most common utilities were built into the shell, a history that let users repeat or modify previous commands easily, and aliases which saved typing on the keyboard. C-shell consists of programming features like arithmetics and C language expression syntax - this is actually where the name came from.

The most serious design flaw of the csh was it's scripting language was significantly different from /bin/sh. At least thousands of Bourne scripts had been written by students and professionals. They were not about to re-hack their scripts. The result was that the original /bin/sh syntax stood. And it stands today.

The version of C-shell of current/modern versions of all BSD's is actually the tcsh shell, an advanced version of the original, csh handles job control, complex aliasing, I/O redirection, command line editing, filename completion, and more.

In 1983 David Korn released a superset version of /bin/sh, the ksh; it was much more flexible than the Bourne shell, though not quite as handy as the csh (it lacked the "!!" and "!$" features, e.g.).

The Korn shell is a superset of the Bourne shellr; that is, it has support for everything that the Bourne shell has. It also offers interactive features as C shell. ksh offers its users programming features such as integer arithemitcs, arrays, functions, aliases. The Korn shell is much faster than the C shell, it is also capable of running Burne shell scripts (compatibility preserved).

Circa 1990, Paul Falstad wrote the first of what we can call the advanced shells, the zsh, or simply zsh. zsh is a superset of the ksh and also had many csh features. The zsh worked with either he scripting language of /bin/sh or ksh, though not, thankfully(!), the csh.

zsh can be used under Windows systems by installing Cygwin, which provides a Unix-like environment for programmers. This is actually the best thing a Unix user can do under Windows - install a Unix shell.

zsh is a shell designed for interactive use, although it is also a powerful scripting language. Many of the useful features of bash, ksh, and tcsh were incorporated into zsh; many original features were added.

bash came into being around this time or shortly afterward; it has followed the same development model as zsh, namely, great power and flexibility while being wholly compatible with the simpler Bourne shell.

Bash is an acronym for Bourne Again SHell (there are already a few stories around on the acronym). All of the Bourne shell builtins are available in Bash, and the rules for evaluation are taken from the POSIX 1003.2 specification for the standard Unix shell. bash is the most popular shell on Linux distros.

Understanding initialization files and environment variables

When a user logs in to the system, the getty(8) program executes login(1). login(1) asks for a username and a password - user identifies himself. If both are correct - authentication was successful, the user is authorized to log into the system and to have his shell. The user's shell is specified in the master.passwd file as the last field in the user's record. Before the shell is given to the user, the login shell looks for and executes two different types of initialization files. The first type are global initialization files and the second are user initialization files. The files are looked up in order and instructions set within are executed. When finished, a shell prompt is displayed on the screen in the left corner.

The shell environment includes variables that are defined by the login(1) program, global initialization file, and the user's initialization files.

There are two types of variables:

• environment variables which provide global information to all programs on user's environment. These variables affect subshells.
• local variables that are valid only under a current shell. Any subshell has no information on those variables.

The following variables are environment ones:

LOGNAME

defines user's login name

HOME

defines user's home, and is a default argument to cd(1) command

SHELL

sets the path to user's shell

PATH

sets the default searching path to find programs

MAIL

sets the user's mailbox

These are local variables:

PWD

current working directory

PS1

shell's prompt

TMPDIR

a directory to place temporary files

System-wide initialization files

A global initialization files, are also called a system-wide initialization files. Those files lie in /etc directory. The two main are /etc/profile and /etc/.login.

Only root is able to maintain those files to provide a global environment for all users on the system.

User initialization files

When the administrator creates a user account, user's template initialization files are copied from /usr/share/skel directory. If you would like to add some custom files or scripts and want them to be copied to the user's home place them in /usr/share/skel. As a sysadmin, you may change them as you wish, but remember that a user will be able redo the changes or make his own.

The goal of user initialization files is that they are used to set user's working environment, such as the path and environment variables. There might be more than one initialization file in user's home. All of them have different meaning and purpose (see the next section).

Initialization files for some shells

Borne (sh)

Example global initialization file: /etc/profile

	MAIL=$HOME/Maildir; export MAIL
	MAIL=/var/mail/$USER; export MAIL
	BLOCKSIZE=K; export BLOCKSIZE

primary user initialization file: $HOME/.profile

Here's an example of a shell function that does a recursive long listing that is piped through more:

	lRm(){ ls -lsR | more }

Variables in the /bin/sh shell are set by using

	VARIABLE=xyz; export VARIABLE

These examples work both in your ~/.profile and from the command line.

	$ foo="bar"; EXPORT foo;
	$ echo $foo
	foo
	
	$ SHELL="/bin/sh"; export SHELL
	$ echo $SHELL
	/bin/sh
	
	$ HOME=/home/user; export HOME
	$ PATH=/bin:/usr/bin:$HOME/bin
	
	$ echo $PATH
	/bin /usr/bin /home/user/bin

As hinted above, you can catenate strings onto non-NULL variables:

	$ PATH=$PATH:/usr/local/bin:/usr/local/sbin:/usr/sbin
	$ echo $PATH
	/bin /usr/bin /home/user/bin /usr/local/bin /usr/local/sbin /usr/sbin

To clear a non-NULL variable without an unset command:

	$ SHELL=""; export SHELL;
	$ echo $SHELL

You may set your variables in ~/.profile file -

	EDITOR=/usr/local/bin/vim
	FCEDIT=/usr/local/bin/vim
	PAGER=/usr/bin/more

and then export it with:

	EXPORT EDITOR FCEDIT PAGER

or simply

	set +o allexport

It is fine to use "set -o allexport" before any definitions (as the very first line in ~/.profile)

secondary user initialization file: n/a

These are read when a new shell (subshell) is started. This isn't applicable in the Bourne shell.

Korn (ksh)

global initialization file: /etc/profile

primary user initialization files: $HOME/.profile and $HOME/.kshrc

The purpose of the $HOME/.profile is the same as for the Bourne shell. To source $HOME/.kshrc file add to your $HOME/.profile:

	ENV=$HOME/.kshrc; export ENV

Korn shell has $HOME/.kshrc file which is the place to set your own aliases, functions or local customizations.

For example:

	# aliases
	alias ls='ls -G'
	alias h='history'
	alias ll='ls -la'
	alias rmi='rm -i'
	alias vi='/usr/local/bin/vim'
	
	# functions
	function lld
	{
		ls -la $* | grep '^d'
	}
	
	function floppyin
	{
		/sbin/mount -t msdos /dev/fd0 /floppy
	}
	
	function floppyout
	{
		/sbin/umount /floppy
	}

secondary user initialization file: $HOME/.kshrc

These are read when a new shell is started. For examples, look above.

C (csh)

global initialization file: /etc/.login

Here you only set globals for your users.

primary user initialization files:$HOME/.cshrc, $HOME/.login and $HOME/.logout

If you use csh, the ~/.login file is executed upon login. There is an analogous ~/.logout file that is executed each time you leave a given instantiation of your shell. In $HOME/.logout file you may put a clear(1) command to cleanup the screen on logout.

Your ~/.cshrc file, which holds aliases and various forms of "set" and "unset", is executed every time you start a csh shell, which may occur more than once in a log in session.

Representative of a fairly simple alias for csh is:

	alias lRm 'ls -lRs | more'

Here are some way of using set commands to initialize a scalar variable:

	% set foo = "bar"
	
	% echo "foo = $foo"
	foo = bar

And to initialize a list to a single variable:

	% set path = ( /{,s}bin /usr/{,s,local/,local/s}bin )
	
	% echo $path
	/bin /sbin /usr/bin /usr/sbin /usr/local/bin /usr/local/sbin

setenv and unenv are used to set (and unset or clear) environment variables.

	% setenv SHELL csh
	% echo $SHELL
	csh

The unsetenv clears the variable 'SHELL'

	    % unsetenv SHELL
	    % echo $SHELL

References:
http://bigworm.colorado.edu/Saclass/class01.html

secondary user initialization file(s): $HOME/.cshrc

These are read when a new shell is started. Look above for the examples.

Z (zsh)

global initialization files:

/etc/zshenv/ holds only global environment variables:

	NNTPSERVER=news.icm.edu.pl
	MAILCHECK=60
	LC_ALL=pl_PL.ISO8859-2
	PROMPT='%B%m%#%b '

/etc/zlogin is sourced in login shells. It should contain commands that should be executed only in login shells.

/etc/zprofile is similar to zlogin, except that it is sourced before zshrc. zprofile is meant as an alternative to zlogin for ksh fans.

/etc/zshrc is sourced in interactive shells. It should contain commands to set up aliases, functions, options, key bindings, etc.

primary user initialization files

$HOME/.zshenv
$HOME/.zprofile
$HOME/.zlogin
$HOME/.zshrc

See the description on those files above.

$HOME/.zlogout: this file is executed during logout - when you type exit, or press ^D.

Local $HOME/.zshenv may consists of:

	PATH=/usr/local/bin:/usr/local/sbin:/usr/X11R6/bin:${HOME}/bin:/bin:/sbin:/usr/bin:/usr/sbin
	EDITOR=/usr/local/bin/vim
	PAGER=/usr/bin/more
	PROMPT='%B%m%#%b '
	RPROMPT="%B[%t%s]%b" - Yes it's possible to have the right fancy prompt
	HISTFILE=~/.history
	HISTSIZE=300

In $HOME/.zshrc you may set your umask:

	umask 027

your aliases:

	alias j='jobs'
	alias h='history'
	alias c='clear'
	alias -g M='| more'
	# The above is an excellent shortcut for "|more". Use as: ls -l M
	alias mutt='/usr/local/bin/mutt'

To search for a user in /etc/passwd file:

	function guser
	{
		/usr/bin/grep $1 /etc/passwd
	}

To look for all directories in current working directory:

	function ld
	{
		ls -F $* | grep '/$'
	}

To grep(1) through running processes:

	psg() { ps auxwww | grep $1 }

To list all unning processes (alias -g M='| more' must be set):

	psm() { ps -auxwww M }

$HOME/.zlogout:
clear - this will clear the screen upon logout.

secondary user initialization file

These are read when a new shell is started: $HOME/.zshrc

Bash (bash)

global initialization file: /etc/profile

This file has the same purpose as for the Bourne shell.

primary user initialization files: $HOME/.bash_profile

.bash_profile is used as the main place to set and export variables, source other files, set umask etc.

For example:

	# Get the aliases and functions
	if [ -f ~/.bashrc ]; then
		. ~/.bashrc
	fi

$HOME/.bash_login as in zsh shell, this file is sourced in login shells.

$HOME/.bash_logout the file is read upon logout, place clear(1) there.

$HOME/.profile .profile can be used instead of .bash_profile.

secondary user initialization file

These are read when a new shell is started.

$HOME/.bashrc .bashrc consists of shell functions and aliases.

TC (tcsh)

The tcsh == csh initialization files are: .login, .logout, and .cshrc. The .log[in|out] files are called once per shell instantiation.

global initialization files: /etc/csh.cshrc, /etc/csh.login, /etc/csh.logout

primary user initialization files: $HOME/.tcshrc, $HOME/.cshrc, $HOME/.login

Sample:

	eval `tset -e^H -i^C -s -Q -m 'dialup:?xterm'`;
	bindkey -v
	MAILCHECK=60
	uptime;

$HOME/.logout: .logout may clear the screen and issue 'fortune' or whatever the user wishes.

secondary user initialization files

These are read when a new shell is started: $HOME/.tcshrc or $HOME/.cshrc.

.cshrc init's env variables, sets the $path and $prompt.

Thanks to us CSHJUNKIES, there is a 1:1 correspondednce between .login and .zlogin, etc.

The second part of this article will be published in next month's Daemon News Answer Man column. Stay tuned!

The AnswerCrew,

Grzegorz Czapliñski
Barry Fox
Gary Kline
Rober Wallis

[ top ]