Courier-IMAP on FreeBSD

Grzegorz Czapliñski <gregory@prioris.mini.pw.edu.pl>

You make your system as secure as you can. You disable Telnet and enable OpenSSH as the only way for your users to reach a shell on your systems. What about other services? Do you secure your SMTP server? What about POP3 and IMAP? You may not implement those services, but then users will have to login via OpenSSH and use Mutt or PINE to read their mail. When you implement them you probably think how to configure them in a secure way. POP3 and IMAP send passwords in clear-text over the network. An attacker may easily steal a user's password sniffing the net or simply hijack his connection. In this article I will show how to install and securely configure Courier-IMAP on your mail server. Your users will have to use secure connections (over SSL) to connect to POP3/SSL and IMAP/SSL services. To understand why it is so important, read this article.

If you are going to move from a non-secure configuration to secure one, you will have to tell your users how to configure their mail clients and warn them when you will remove old services. That will give you less trouble with users or even more - that will make you more user friendly... Otherwise they will send you emails and keep asking over and over again how to change this or that.

What is Courier-IMAP?

Courier's README describes it the best:

Courier-IMAP is a server that provides IMAP access to Maildirs. This IMAP server does NOT handle traditional mailbox files (/var/spool/mail, and derivatives), it was written for the specific purpose of providing IMAP access to Maildirs.

"Maildir" is a directory-based mail storage format originally introduced in the Qmail mail server and adopted as an alternative mail storage format by both Exim and Postfix. Courier-IMAP is popular on Qmail/Exim/Postfix sites that are configured to use maildirs. The primary advantage of maildirs is that multiple applications can access the same Maildir simultaneously without requiring any kind of locking whatsoever. It is a faster and more efficient way to store mail. It works particularly well over NFS, which has a long history of locking-related woes.

For more information visit: http://www.inter7.com/courierimap.html

If you use Postfix as your MTA and want to use Maildirs instead of mailboxes and still want to use Courier-IMAP read on.

Courier-IMAP installation

To install courier go to /usr/ports/mail/courier-imap (I assume you have a freshly cvsuped ports collection); check your umask (it should be set to 022) and simply type:

	# make
	# make install

The installation will take some time so sit back and relax.

Courier configuration

Don't start anything yet! Wait a moment! :-)

By default the courier-imap port installs two sample start/stop scripts to /usr/lcoal/etc/rc.d. We will not use them as they are links to the /usr/local/libexec/courier-imap/ directory:

	# ls -l /usr/local/etc/rc.d/courier-imap-*
	lrwxr-xr-x  1 root  wheel  40 Jun 19 08:34 /usr/local/etc/rc.d/courier-imap-imap
	d.sh.sample -> /usr/local/libexec/courier-imap/imapd.rc
	lrwxr-xr-x  1 root  wheel  40 Jun 19 08:34 /usr/local/etc/rc.d/courier-imap-pop3
	d.sh.sample -> /usr/local/libexec/courier-imap/pop3d.rc

You can remove those links for now.

Courier-IMAP is installed with several authentication modules. I will list some of them here.

authpwd

This module looks up userids and passwords in your /etc/passwd file or the equivalent NIS map (as supported by your system's getpw library).

authshadow

This module is like authpwd, except that it should be used on systems that use shadow password files, /etc/shadow.

authpam

This module should be used on systems that have the PAM library.

Additionally: PAM is used only for password validation. The home directory and the user and group IDs for the account are still taken from the system password file.

authuserdb

This module uses GDBM or DB database files, usually /etc/userdb.dat and /etc/userdbshadow.dat to look up userids and passwords.

authldap

Authenticates against an LDAP server. This is a new module included with Courier-IMAP.

authmysql

Native MySQL authentication module.

authpgsql

Native PostgreSQL authentication module.

authdaemon

Background daemon authentication proxy. This module is always compiled by default.

authcustom

This is a dummy authentication module that does not authenticate anything. It is a placeholder that can be used to implement any site-specific authentication code, written in C.

By default Courier-IMAP from ports under FreeBSD is compiled with authpam, authuserdb, authdaemon (authcustom) modules.

Selecting the authdaemon module (it is selected by default) will compile and install a separate process, "authdaemond". The "authdaemond" process is started and stopped by the imapd.rc, imapd-ssl.rc, pop3d.rc and pop3d-ssl.rc.

ATTENTION: all four scripts stop authdaemond. Typically all the scripts are called together, at system startup and shutdown. If you need to stop just one service, find the pid and kill it manually. Otherwise authdaemond stops and nobody can log in anymore.

The authdaemon module offers an alternative to compiling all the authentication modules as standalone ones. Enabling authdaemon in addition to any other modules will result in authdaemon being built as the only "official" authentication module. All other modules are compiled into a separate process, "authdaemond".

In /usr/local/etc/courier-imap/ there is a file called authdaemonrc.dist. This is a configuration file used by the authdaemond process. Change its name to authdaemonrc and eventually tweak the config.

	authmodulelist="authcustom authuserdb authpam"

This paramters specifies a list of authentication modules that are linked into authdaemond. If you autheticate against PAM use authpam.

	daemons=5

This is the max number of daemons to run.

In fact /usr/local/libexec/courier-imap/authlib/authdaemond is a script. Depending on your system configuration, there may be one or more different authdaemond binaries installed, and here's why:

authdaemond.plain will include all available authentication modules except for certain "heavy" authentication modules. The current list of "heavy" authentication modules is authldap, authpgsql and authmysql.

If support for a "heavy" authentication module is selected, there will be an additional binary installed, such as authdaemond.mysql, authdaemond.pgsql or authdaemond.ldap. The /usr/local/libexec/courier-imap/authlib/authdaemond script checks if any "heavy" authentication daemon is installed, and, if so, runs that. Otherwise, the default authdaemond.plain binary goes in.

Read this script with more(1) and you will understand more.

Next, go to /usr/local/etc/courier-imap and edit imapd.cnf.dist and pop3d.cnf.dist. Edit "[ req_dn ]" only, then change names of those files to imapd.cnf and pop3d.cnf.

The [ req_dn ] should look like:

	[ req_dn ]
	C=countryName Two letters!
	ST=stateOrProvinceName
	L=localityName
	O=organizationName
	OU=OrganizationalUnitName
	CN=commonName
	emailAddress=emailAddress

Mind that the "C" tag is for country and only two letters long, like: C=PL, or C=US.

Execute these commands:

	# /usr/local/sbin/mkimapdcert
	# /usr/local/sbin/mkpop3dcert

These commands will create self-signed certificates for your IMAP and POP3 servers. By default the certificates will be saved in /usr/local/share/courier-imap/ directory under imapd.pem and pop3d.pem names. Make sure the access rights -rw are only for root, group wheel!

If you need to specify any options to your IMAP/SSL server edit imapd-ssl.dist. To change POP3/SSL edit pop3d-ssl.dist. Change its names to imapd-ssl and pop3d-ssl respectively.

You might be interested in changing just a few variables.

SSLADDRESS

is responsible for the IP address assigned to an interface your IMAP or POP3 server will listen on. By default it listens on all interfaces.

TLS_CERTFILE

specifies the place where you keep your certificate.

TLS_VERIFYPEER

is set to NONE. You may change it to PEER or while being more restrictive to REQUIREPEER. PEER option tells IMAP/SSL to verify the client's certificate, if one's presented. REQUIREPEER - requires a client to have a certificate; authentication fails if one's not presented.

Do the same with imapd.dist and pop3d.dist. Change their names to imapd and pop3d. If needed tweak the config.

Now go to /usr/local/etc/rc.d and make startup scripts:

	# cd /usr/local/etc/rc.d
	# ln -s /usr/local/libexec/courier-imap/imapd-ssl.rc imapd-ssl.sh
	# ln -s /usr/local/libexec/courier-imap/pop3d-ssl.rc pop3d-ssl.sh

Now it's high time to check how our config is working.

Try:

	# /usr/local/etc/rc.d/imapd-ssl.sh start
	# /usr/local/etc/rc.d/pop3d-ssl.sh start

Using sockstat check if something is listening on ports 993 and 995.

	# sockstat -l4 
	USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN
	ADDRESS
	root     couriertcp 64541 5  tcp4   *:995                 *:*
	root     couriertcp 64522 5  tcp4   *:993                 *:*
	...

To test if SSL protocol is working try these commands:

	openssl s_client -host imap-server -port port -verify -debug -no_tls1
	openssl s_client -host imap-server -port port -verify -debug -no_ssl2
	openssl s_client -host imap-server -port port -verify -debug -no_ssl3

Set imap-server to your_imap-server.domain.com and port to 993 or 995.

This is the easiest method to check if SSL is working fine.

Postfix configuration

To make Postfix deliver to Maildirs instead of mailboxes, edit main.cf file. Find the section marked with "# DELIVERY TO MAILBOX" and change "home_mailbox = Mailbox" to "home_mailbox = Maildir/".

Save changes and issue postfix reload command. To check issue:

	# postconf -n | grep Maildir
	home_mailbox = Maildir/

All is fine, postfix will deliver to Maildirs now. Great!

If your system is brand new - you are just setting it up, and there are no users yet you are lucky. If you change the configuration with existing users that might be a pain. What you have to do now is to convert their mailboxes to Maildirs.

Mbox2Maildir conversion

Stop your mailing system with:

	# postfix stop

For every user on the system issue a command:

	# /usr/local/bin/maildirmake ~user/Maildir

This command will create a Maildir directory in users' home. Download the mbox2maildir perl script from http://www.qmail.org/mbox2maildir. I would advise to apply this patch http://asdf.dk/mail/mbox2maildir.diff. This will preserve your old mboxes.

	# patch -p < mbox2maildir.diff
	# chmod 700 mbox2maildir

To convert one mailbox simply issue:

	# ~/bin/mbox2maildir ~/mbox ~/Maildir 

That will convert the mbox file to Maildir format. You can check it with:

	# ls -l ~/Maildir/new
	total 8
	-rw-r--r--  1 root  wheel  2787 Jun 19 17:47 1056037672.64721.mbox
	-rw-r--r--  1 root  wheel   589 Jun 19 17:47 1056037673.64721.mbox
	-rw-r--r--  1 root  wheel  1446 Jun 19 17:47 1056037674.64721.mbox

Do the same for every file in ~/Mail directory; or simply:

	# cd ~/Mail
	# for i in *; do ~/bin/mbox2maildir ~/Mail/${i} ~/Maildir/.${i} ; done

The problem (not a big one) occurs when you have subfolders.

	# ls -l ~/Mail
	total 8
	-rw-------  1 root  wheel  4996 Jun 19 17:51 files
	drwxr-xr-x  2 root  wheel   512 Jun 19 17:51 mail-lists

Here, 'files' is a simple mbox format file which gets converted by the command above. 'mail-lists' is a folder with two mbox format files - maillist1 and maillist2. They will not get converted so you have to modify the above command.

First descend one level:

	# cd mail-lists
	# for m in *; do ~/mbox2maildir ~/Mail/mail-lists/${m} ~/Maildir/.mail-lists.${m}; done

Your Maildir will look like:

	alphax% ls -la ~/Maildir
	total 16
	drwx------   8 root  wheel   512 Jun 19 18:17 .
	drwxr-xr-x  10 root  wheel  1024 Jun 19 18:16 ..
	drwx------   5 root  wheel   512 Jun 19 18:03 .files
	drwx------   5 root  wheel   512 Jun 19 18:31 .mail-lists.maillist1
	drwx------   5 root  wheel   512 Jun 19 18:31 .mail-lists.maillist2
	drwx------   2 root  wheel   512 Jun 19 17:25 cur
	drwx------   2 root  wheel   512 Jun 19 17:47 new
	drwx------   2 root  wheel   512 Jun 19 17:25 tmp

You may wonder what the '.' are doing there in the names of the folders.

First, some IMAP servers put all of your mail into a subfolder, often a subfolder of INBOX. If your server is Courier or Cyrus, your folder Lists may actually be INBOX.Lists. Courier and Cyrus use '.' to separate folder names. UW-IMAP uses '/'; Lotus Notes in contrast uses '\'.

In Mutt you have to use this path to call your maillist1 Maildir:

	imap://hostname/INBOX.mail-lists.maillist1

When you have finished the conversion from mbox to Maildir, congratulations, you are done! :)

Restart postfix and check if you can read your mail via IMAP/SSL and POP3/SSL.

If you use mutt as your client of choice edit the .muttrc file and add:

	set folder=imaps://localhost/INBOX
	set mask="\..*"
	set spoolfile=imaps://localhost:993/INBOX
	set mbox=imaps://localhost:993/INBOX
	set imap_user="gregory"
	#set imap_pass="xxxxxxx"
	set ssl_use_sslv3
	set certificate_file="~/.mutt.certificate"

To read your mail locally, you can connect to IMAP without SSL support. If so, create a startup script:

	# ln -s /usr/local/libexec/courier-imap/imapd.rc
	/usr/local/etc/rc.d/imapd.sh

You may edit /usr/local/etc/courier-imap/imapd and set "ADDRESS=127.0.0.1" so imapd will listen only on that address, or firewall port 143.

If you connect to IMAP, remember to change "imaps" to "imap" above. The mask is important here! Setting certificate_file will tell mutt to store the server's certificate. Otherwise mutt will ask you to accept the cert everytime you connect to the server.

To see how configure other GUI clients, read: http://karmak.org/2003/courier-imap/.

Some additional notes:

• If you want to use webmail, for example SquirrelMail, you must have IMAP enabled on port 143.

  In my configuration I have IMAP listening on ports 143, 993 and 995, but only the last two are accessible from the outside. Port 143 is blocked by my firewall.

• You can create a Maildir directory in /usr/share/skel. When adding a new user pw will copy all files from that directory to the new user's home.

  	# maildirmake /usr/share/skel/Maildir
  

• If you filter your mail with procmail you can still use it with Maildirs. The alternative (maybe better) is a maildrop program. You can install it from the ports collection. Read Postfix's MAILDROP_README to learn how to set it up.

  To use procmail edit .procmailrc and tweak as follows:

  	PATH=/bin:/usr/bin:/usr/local/bin
  	MAILDIR=$HOME/Maildir
  	DEFAULT=$HOME/Maildir/
  	MAILLISTFOLDER=$MAILDIR/.mail-lists
  	
  	# mail from postfix
  	:0
  	 * ^(To|Cc):.*postfix-users
  	 $MAILLISTFOLDER.postfix-users/
  

  In this example all mail containing the "postfix-users" string in the To: or Cc: header will be delivered to ~/Maildir/.mail-lists.postfix-users/

• (How) Can I use Pine with Maildirs?

  It is possible. First you have to download a patch for Pine. Go to http://hico.fphil.uniba.sk/pine-patches.html, download the IMAP patch, apply it and recompile. Apply the patch to Pine from Ports as it needs some other patches to work under FreeBSD.

  Then, set this in /etc/profile:

  	MAIL=$HOME/Maildir; export MAIL
  

Setting up Courier-IMAP with SSL is not difficult. It takes some time, but you will benefit. This will make your mailing system secure and resistant to spoofing and sniffing. Your users will retrive their mail in a secure way. What more can I say? Configure it yourself!

Links

• http://www.inter7.com/courierimap.html
• http://www.asdf.dk/text/mbox2maildir.html
• http://karmak.org/2003/courier-imap/
• http://mutt.sourceforge.net/imap/