Switching from MH and Evolution to maildir, Dovecot IMAP and Thunderbird

Updated. New details added.

I got fed up with Evolution recently. My main beef with it is that it's all about magic on the back end. All sorts of details, such as what you've deleted or what you've marked as spam are flagged using proprietary metadatabases. The persistence of this information also seems to be dodgy, which means that often, when Evolution crashes it forgets such details, and makes you re-delete, re-file and re-flag things. To make this worse, Evolution crashes a lot, and it seems to especially crash after you've done a lot of message deleting and moving. Over time, Evolution has cost me a lot of tedious work (I get a lot of spam and throwaway mail) and I got to the point where I could no longer stand a tool that increased my workload for little gain.

The main reason I stuck with Evolution for so long is that I have used the MH system to manage my mail for a long time. I like the idea of one message per file. Evolution was the only very modern MUA I could find that supported MH folders. I'd used EXMH for a long time, but it's showing its age, and Sylpheed-claws has way too many rough edges.

So it was time for a change on my Ubuntu workstation, and this is just my notes about what it took to complete the switch of mail management systems.

MH to Maildir

In the end I decided to switch to Maildir, which is like MH in its one-file-per-message philosophy, but seems to be supported by more modern MUAs. I also decided to hedge my bets by moving to a local IMAP server to expose the Maildir folders to just about any MUA.

For the main conversion I used Dr. Jürgen Vollmer's mh2maildir. It did the trick very well. All my MH folders were stored in a directory Mail, and I wanted to have the resulting Maildir in .maildir. I invoked the script as follows:

./bin/mh2maildir.sh -r -R ~/Mail/ ~/.maildir

I knew I had no unread e-mail, thus the -r option. Most users will need the -R option. The original MH diectory is left alone, as far as I can tell, but of course I always recommend backing up your mail folders before any bulk action.

Update. You may need to make some structural adjustments in the resulting .maildir folder. Firstly, many MH tools use an explicit folder named "inbox", which is thus converted into a Maildir folder named "inbox". Most Maildir tools, however, treat the top folder as the implicit inbox. I addressed this as follows:

cd ~/.maildir
mv tmp tmp.save
mv inbox/* .
rm -rf inbox

Also, some Maildir tools such as Mutt and Dovecot expect to see folders as hidden filees (with a leading dot). I handled this by renaming folders, for example:

mv Writing/ .Writing/

If you have nested folders, you have to use a dot to separate levels. A folder "eggs" within "spam" is represented by a top-level directory called ".spam.eggs". I had to do the likes of:

mv XML/XML-DEV .XML.XML-DEV
mv XML .XML

Of course leave alone the "cur", "new" and "tmp" directories which make up the internal structure of each Maildir folder.

Dovecot for IMAP

I went with the Dovecot server for IMAP. I just used Synaptic to install dovecot-common and dovecot-imapd, and followed the article "Setting up an IMAP server with dovecot". The trickiest line in setting up /etc/dovecot/dovecot.conf seems to be the default_mail_env.

default_mail_env = maildir:/home/%u/.maildir

Update. I also had to enable imap with the lines:

protocols = imap
imap_listen = *

And for some reason I had to manually start dovecot the first time:

sudo dovecot

After this, the usual sudo /etc/init.d/dovecot restart seems to work.

Thunderbird

The thunderbird set-up was very straightforward. In fact, I can't really think of anything to say about it. Just set it up with localhost as an IMAP server, and you're set.

Finally, updating .fetchmail

I use fetchmail to download my mail from my external IMAP server, and procmail to put it into my local folders. I had to change procmail to work with Maildir rather than MH folders. The changes turned out to be very simple. In my .procmailrc having defined MAILDIR=$HOME/.maildir I changed lines such as

|/usr/lib/mh/rcvstore +Writing

to

$MAILDIR/.Writing/

Nested folders are similar. You use periods rather than slashes for level separators, not forgetting the leading period. Thus

|/usr/lib/mh/rcvstore +Writing/Tech

becomes

$MAILDIR/.Writing.Tech/

Since Mutt/Dovecot don't use explicit "inbox" folders,

|/usr/lib/mh/rcvstore +inbox

becomes just

$MAILDIR/

The trailing slashes are essential. They indicate that the destination is a Maildir folder and so procmail does the right thing with it.

Mutt is useful for testing that all is OK with your IMAP set-up. Use regular slashes rather than periods for folder separators in IMAP urls, for example:

mutt -f imap://uogbuji@localhost/Writing/Tech

[Uche Ogbuji]

via Copia
5 responses
<blockquote type="cite" cite="http://copia.ogbuji.net/blog/2006-02-01/Switchi... was the only very modern MUA I could find that supported MH folders.</blockquote>



mutt supports MH folders, and has years.  Any reason you decided not to go with mutt?  Not "modern" enough for ya? ;)
I do use Mutt, on shell accounts when I log in.  I'm not crazy about it, and no, it's not really modern enough for me.  "Modern" is of course a subjective term.  I prefer command line and emacs for a lot of things, which many think are not modern.  I prefer the likes of Thunderbird for Mail.  I prefer the Total War games to MUDs.  I prefer John Donne to Walt Whitman.  I prefer Yule to Christmas.  I prefer calculators to slide rules.
Some useful info about MailDir variations:



http://wiki.mutt.org/?ExtendedMaildirFormat



Getmail--a fetchmail alternative I considered but left alone for now:



http://pyropus.ca/software/getmail/
Just a quick comment on mh2maildir and the Folder/Subfolder/SubSubFolder -> Folder.Subfolder.SubSubFolder renaming...



If you use the -courier switch to mh2maildir (at least as of rev 1.13 which I just downloaded), it will do that renaming for you.  You'll still need to move the inbox folder to the root of your Maildir, however.



I'm currently hacking mh2maildir a little and will probably fix the above and (more importantly) make it understand MH read/unread status so that it can auto-assign messages to one of read or unread status in the Maildir and will then send the changes back Dr. Jürgen Vollmer for his consideration.  With those changes, I should finally be able to move my email somewhat into the modern era (and amortize that Dovecot server that's running for my wife's email anyway ;)).



Thanks for this page, btw... I always dreaded the conversion process, but seing a cookbook recipe and scripts to do the dirty work helped get me moving...
great tutorial, thanks. here's a script i run after mh2maildir to get all the folder names changed.

#!/bin/sh

# maildir-fixup.sh run this after mh2maildir to fix up the folder names
# vixie 11jun2011 [new]

cd ~/Maildir || exit
mkdir -p tmp cur new
find * -depth \
-type d ! -name '.*' ! -name cur ! -name new ! -name tmp -print | \
while read old; do
new=.$(echo $old | sed -e 's/\./_/g' -e 's/\//./g')
printf "%-30.30s %s\\n" $old $new
mv $old $new
done

exit