Ubuntu Edgy, Firefox and Flash

One of the things I noticed upon upgrade to Edgy is that I'd lost Flash support in Firefox. I wish I could say that I had no need for Adobe/Macromedia's crap, but I can't. For one thing, I often make musical discoveries on MySpace pages--my musical interests tend towards the underground. I set about this evening getting Flash working again, and it was harder than I expected. Recently MySpace switched to Flash 9 (punks!) and I decided to use the version 9 beta bone that Adobe tossed to the Linux community. I found that the nonfree virtual Flash package had ben removed, so I restored it:

sudo apt-get install flashplugin-nonfree

The plan would be to just copy the libflashplayer.so from Flash 9 beta over the installed version 7. BTW the Flash 7 file was 2MB and the Flash 9 Beta file is almost 7MB. Might be debugging info or something, but yeesh!

Anyway, even before copying the Flash 9 beta I found that Firefox was crashing every time it loaded a Flash site. After some hunting I found a bug with Ubuntu Edgy and either version of the Flash plugin. There are several workarounds mentioned in that thread, but one of them is simply to bump up color depth to 24 bits. I hadn't even known Edgy had limited me to 16 bits, but surely enough I checked my /etc/X11/xorg.conf:

Section "Screen"
    Identifier      "Default Screen"
    Device          "NVIDIA Corporation NV34M [GeForce FX Go 5200]"
    Monitor         "Generic Monitor"
    DefaultDepth    16

I changed that from 16 to 24, restarted X, and all has been well since then. Fingers crossed.

Meanwhile, speaking of music on MySpace, here's my brother's crew, StrawHat BentLow.

...your best advice is look for the hat...

See also: "On Huck, Hip Hop, and Expression"

[Uche Ogbuji]

via Copia

From Dapper to Edgy

I upgraded my Dell 8600 laptop from Dapper to Edgy tonight. I waited a bit because I've heard a lot of problem reports, and as a developer, I tend to trick out my machine config, and tricked out machine configs are usually especially vulnerable to upgrade problems. I do want to take a moment to mention that people who label any complaint about the upgrade process as F.U.D. are quite out of order. Your upgrade might have gone smoothly, but that does not invalidate warnings fro folks whose upgrades didn't. These people are doing a valuable service by making users cautious about any upgrade. That having been said, my own upgrade has been very smooth so far. Herein a few notes others might find useful.

I prepped the install by following the suggestion on this SlashDot thread. I ran a query for packages no longer available from any of my repositories:

apt-show-versions | awk '/No available/ { print $1 }'

Then after inspecting the results and not finding anything I was desperate to keep, I nuked them all:

apt-show-versions | awk '/No available/ { print $1 }' | sudo xargs dpkg --remove

I did have to install the apt-show-versions command first. Then I just followed the official upgrade instructions. The first attempt aborted with

Error during update...A problem occurred during the update. This is usually some sort of network problem, please check your network connection and retry.

It turned out that I had stuff in /etc/apt/sources.list that has become unavailable (stuff from Penguin Liberation Front and Cipherfunk). I just edited the file and uncommented the problem lines, but I think they need to fix how the installer handles this situation. A newbie might indeed think he has a problem with his network, and go on a wild goose chase. The error message should at least mention the possibility that certain remote repositories are having trouble, and maybe even offer to disable the problem repositories for the user. As for the PLF stuff the instructions suggest touching up the sources line to:

deb http://packages.freecontrib.org/ubuntu/plf/ edgy-plf free non-free

My first impression of the new version is that it looks even nicer than Dapper. I'm deciding whether I like the aggressive anti-aliasing in gnome-terminal. I don't mind AA in general, but something about a terminal or programming editor makes me prefer sharpness over slickness. The new fonts are definitely slick in Firefox.

I'm yet to upgrade all the multimedia/non-free stuff. I let the upgrade nuke all that (something I highly recommend). I used to use Automatix (check that slick new Web site), but these days the restricted formats Wiki is good enough for me not to need any magic tools.

Based on my positive experience (so far), I'll upgrade my main desktop tomorrow. Thanks, as always, to the Ubuntu folks.

[Uche Ogbuji]

via Copia

Just a friendly game of mailbox baseball

Baseball was never for Blacks
It used to be a pastime for Whites
Now it has mad Puerto Ricans
But that's not the point of this song.
The point of this song, and I make it mad simple when I be flipping this script
Is that the industry is all over the mound pitching but nobody's making any hits.
Hmmmmmm.

—Natural Resources—Negro League Baseball

Yeah. Mailbox baseball. That game of legend (I've never seen it played) where a pack of stereotypical American teenage lombards drives down the boulevard whacking at mailboxes with Louisville sluggers from the car windows until they hit one with unclaimed mail, enjoying the resulting shower of fluttering letters. I did something like that to myself today, quite unwittingly. (I'm amazed there's room in my cheek for the tongue after a day like today).

I've been completely retooling my e-mail habits, in part to use uche@ogbuji.net more for professional correspondence that does not have immediate bearing on my day job. I the process I managed to completely kill my fourthought.com address today, and as I hear it, it's all been bouncing to hell. Sorry folks. Unless you work for a company that writes checks to Fourthought, Inc., I'll probably be asking you politely to start using my ogbuji.net address from now on anyway, so you might as well start now (that address wasn't affected by the outage). Meanwhile I've straightened out the config problems, and as soon as DNS has a chance to propagate, my fourthought.com address should be working again.

BTW, those who have ever sent me e-mail at my ogbuji.net account and learned how infrequently I get around to checking it will notice a marked improvement in my attention (I should temper that promise by mentioning that I haven't been able to keep up with even my fourthought.com address in years. It's no fun being afflicted by chronic Scrolliosis).

Back-to-off-topic note w.r.t. my silly intro: If you're an underground hip-hop head and haven't heard the song whose lyrics I used you'd best get up out there and find that single. It's a classic from 1997, and it was the first time I heard the M.C. then known as "What What?" and now recognized as the great Jean Grae (repping South Africa—she runs through your hood with her middle finger up.). It's a great romp of a song that refuses to take itself too seriously, and has the tickly lounge piano in the background loop to match. It's da Jawn!

[Uche Ogbuji]

via Copia

Adding feeds to Liferea on the command line

Despite the kind help of the Rojo people I still can't get the service to import my updated feed lists ('An error has occurred...Failed to import: null...We apologize for the inconvenience.'), so I'm still reading my Web feeds on Liferea for now. One nice bonus with Liferea is the ability to add feeds from the command line (or really, any program) courtesy GNOME's DBUS. Thanks to Aristotle for the tip, pointing me to 'a key message on liferea-help'. I've never used DBUS before, so I may be sketchy on some details, but I got it to work for me pretty easily.

I start with a simple script to report on added feed entries. It automatically handles feed lists in OPML or XBEL (I use the latter for managing my feed lists, and Liferea uses the former to manage its feed list).

import amara
import sets

old_feedlist = '/home/uogbuji/.liferea/feedlist.opml'
new_feedlist = '/home/uogbuji/devel/uogbuji/webfeeds.xbel'

def get_feeds(feedlist):
    doc = amara.parse(feedlist)
    #try OPML first
    feeds = [ unicode(f) for f in doc.xml_xpath(u'//outline/@xmlUrl') ]
    if not feeds:
        #then try XBEL
        feeds = [ unicode(f) for f in doc.xml_xpath(u'//bookmark/@href') ]
    return feeds

old_feeds = sets.Set(get_feeds(old_feedlist))
new_feeds = sets.Set(get_feeds(new_feedlist))

added = new_feeds.difference(old_feeds)
for a in added: print a

I then send a subscription request for each new item as follows:

$ dbus-send   --dest=org.gnome.feed.Reader /org/gnome/feed/Reader \
  org.gnome.feed.Reader.Subscribe \
  "string:http://feeds.feedburner.com/DrMacrosXmlRants"

The first time I got an error "Failed to open connection to session message bus: Unable to determine the address of the message bus". I did an apropos dbus and found dbus-launch. I added the suggested stanza to my .bash_profile:

if test -z "$DBUS_SESSION_BUS_ADDRESS" ; then
    ## if not found, launch a new one
    eval ‘dbus-launch --sh-syntax --exit-with-session‘
    echo "D-BUS per-session daemon address is: $DBUS_SESSION_BUS_ADDRESS"
fi

After running dbus-launch the dbus-send worked and Liferea immediately popped up a properties dialog box for the added feed, and stuck it into the feeds tree at the point I happened to last be browsing in Liferea (not sure I like that choice of location). Simple drag&drop to put it where I want. Neat.

[Uche Ogbuji]

via Copia

Switching from MH and Evolution to maildir, Dovecot IMAP and Thunderbird

Updated. New details added.

I got fed up with Evolution recently. My main beef with it is that it's all about magic on the back end. All sorts of details, such as what you've deleted or what you've marked as spam are flagged using proprietary metadatabases. The persistence of this information also seems to be dodgy, which means that often, when Evolution crashes it forgets such details, and makes you re-delete, re-file and re-flag things. To make this worse, Evolution crashes a lot, and it seems to especially crash after you've done a lot of message deleting and moving. Over time, Evolution has cost me a lot of tedious work (I get a lot of spam and throwaway mail) and I got to the point where I could no longer stand a tool that increased my workload for little gain.

The main reason I stuck with Evolution for so long is that I have used the MH system to manage my mail for a long time. I like the idea of one message per file. Evolution was the only very modern MUA I could find that supported MH folders. I'd used EXMH for a long time, but it's showing its age, and Sylpheed-claws has way too many rough edges.

So it was time for a change on my Ubuntu workstation, and this is just my notes about what it took to complete the switch of mail management systems.

MH to Maildir

In the end I decided to switch to Maildir, which is like MH in its one-file-per-message philosophy, but seems to be supported by more modern MUAs. I also decided to hedge my bets by moving to a local IMAP server to expose the Maildir folders to just about any MUA.

For the main conversion I used Dr. Jürgen Vollmer's mh2maildir. It did the trick very well. All my MH folders were stored in a directory Mail, and I wanted to have the resulting Maildir in .maildir. I invoked the script as follows:

./bin/mh2maildir.sh -r -R ~/Mail/ ~/.maildir

I knew I had no unread e-mail, thus the -r option. Most users will need the -R option. The original MH diectory is left alone, as far as I can tell, but of course I always recommend backing up your mail folders before any bulk action.

Update. You may need to make some structural adjustments in the resulting .maildir folder. Firstly, many MH tools use an explicit folder named "inbox", which is thus converted into a Maildir folder named "inbox". Most Maildir tools, however, treat the top folder as the implicit inbox. I addressed this as follows:

cd ~/.maildir
mv tmp tmp.save
mv inbox/* .
rm -rf inbox

Also, some Maildir tools such as Mutt and Dovecot expect to see folders as hidden filees (with a leading dot). I handled this by renaming folders, for example:

mv Writing/ .Writing/

If you have nested folders, you have to use a dot to separate levels. A folder "eggs" within "spam" is represented by a top-level directory called ".spam.eggs". I had to do the likes of:

mv XML/XML-DEV .XML.XML-DEV
mv XML .XML

Of course leave alone the "cur", "new" and "tmp" directories which make up the internal structure of each Maildir folder.

Dovecot for IMAP

I went with the Dovecot server for IMAP. I just used Synaptic to install dovecot-common and dovecot-imapd, and followed the article "Setting up an IMAP server with dovecot". The trickiest line in setting up /etc/dovecot/dovecot.conf seems to be the default_mail_env.

default_mail_env = maildir:/home/%u/.maildir

Update. I also had to enable imap with the lines:

protocols = imap
imap_listen = *

And for some reason I had to manually start dovecot the first time:

sudo dovecot

After this, the usual sudo /etc/init.d/dovecot restart seems to work.

Thunderbird

The thunderbird set-up was very straightforward. In fact, I can't really think of anything to say about it. Just set it up with localhost as an IMAP server, and you're set.

Finally, updating .fetchmail

I use fetchmail to download my mail from my external IMAP server, and procmail to put it into my local folders. I had to change procmail to work with Maildir rather than MH folders. The changes turned out to be very simple. In my .procmailrc having defined MAILDIR=$HOME/.maildir I changed lines such as

|/usr/lib/mh/rcvstore +Writing

to

$MAILDIR/.Writing/

Nested folders are similar. You use periods rather than slashes for level separators, not forgetting the leading period. Thus

|/usr/lib/mh/rcvstore +Writing/Tech

becomes

$MAILDIR/.Writing.Tech/

Since Mutt/Dovecot don't use explicit "inbox" folders,

|/usr/lib/mh/rcvstore +inbox

becomes just

$MAILDIR/

The trailing slashes are essential. They indicate that the destination is a Maildir folder and so procmail does the right thing with it.

Mutt is useful for testing that all is OK with your IMAP set-up. Use regular slashes rather than periods for folder separators in IMAP urls, for example:

mutt -f imap://uogbuji@localhost/Writing/Tech

[Uche Ogbuji]

via Copia

The madness of Samba setup on Ubuntu

Yeah, Ubuntu has thrown a few medium-sized annoyances at me recently. Tonight I wanted to set up Samba for sharing files with our local Windows machines. I've set up Samba dozens of times, but this is one of the more confusing episodes I've come across recently.

First of all I looked and saw that I had a package named "samba-common", and yet this did not mean I actually had Samba. A little confusing, but I just had to fire up Synaptic and search for "samba". So I installed the package and figured I'd have a serviceable default config. No dice. Rather than hack at /etc/smb.conf I decided to use swat, the Web control panel for Samba, which I've used, and I like. I installed it and tried to surf http://localhost:901/, the usual way to access swat. No response. I read the swat man page which said it runs through inetd. The man page specifically mentions editing /etc/inetd.conf I looked there and found but two lines, one of which was:

#<off>#swat            stream  tcp     nowait.400      root    /usr/sbin/tcpd  /usr/sbin/swat

How quaint! Old school inetd. How odd! Only 2 managed services. Oh well. I uncommented the line and then tried to restart it. Umm. no trace of inetd beside that config file.

At this point I did some googling. I just wanted a simple Ubuntu-specific HOWTO. I wasn't really in much of a mood for pulling out all the hacking stops. Google had almost nothing to offer except others bitching about how hard Samba is to set up on Ubuntu. I went back to work. I checked Synaptic and it turned out I had no inetd package installed. I selected xinetd, which is what I'd been expecting in a modern distro, anyway.

Now I had to configure swat to run behind it, but I was on my own. This is why a package such as swat should depend on a package such as xinetd, so that it can provide a plug-and play config. I'm pretty sure it does on Fedora Core. And certainly there shouldn't be phantom inetd.conf files floating around to trip up users.

I created /etc/xinetd.d/swat, with these contents:

# description: SAMBA SWAT
service swat
{
    disable         = no
    socket_type     = stream
    protocol        = tcp
    user            = root  #should use a more limited user here
    wait            = no
    server          = /usr/sbin/swat
}

And then dpkg-reconfigure xinetd to restart xinetd (and do any other magic needed after updating config). Finally I got swat up. If you want to use swat to administer Samba, you'll need a proper root password. No sudo to bail you out. If you log in as your user, you get a crippled, pee-on swat console for mere mortal users. On one machine I got my Ubuntu root by booting to single user mode and then setting the password. On this machine I used Automatix. Either way, use that root login to get the full plate of admin options. From that point it's fairly smooth sailing. swat isn't perfect, but it's as close as you get to simple Samba administration.

I hope this helps someone else. Maybe it's my imagination, but this seems to be much harder than I remember it ever being on Fedora Core or Mandrake. Oddly enough, I'm getting the impression that Ubuntu rocks for user desktop stuff, but gets really clumsy when it comes to the server management goods that should be bread and butter for Linux.

[Uche Ogbuji]

via Copia

Using .signature in Evolution

I'm just about fed up with Evolution by now, and I'm looking for ways to loosen my dependency from the monster (see upcoming entry). It tries to be way too smart with your data and makes some basic things matters for hair-pulling. One example concerns e-mail signatures. I have a ~/.signature, as has been the convention since the beginnings of days. I've found it ridiculously hard to get Evolution to work with that. Evolution 2.x allows you to paste in a signature, or set up a script to generate signatures, but not to use an existing file. I tried hacking up a script along the lines of cat ~/.signature but that didn't work because Evolution expects you to use <br> in your script output rather than newlines. WTF! I not using HTML mail, damn it! Yeah, I could hack around that using sed and all that, but I shouldn't have to. After some googling and experimentation I finally figured out a workaround recipe.

1) Create a signature using Edit -> Preferences and then `Composer Preferences -> Signatures -> Add. Give it the namesymlink .sig` and put some easily spotted but inconsequential text in the body. Save it.

2) grep foo ~/.evolution/signatures/* where you replace "foo" with your dummy body text from the last step. This will tell you the file in which Evolution saved the dummy signature.

3) Clobber the dummy signature with a symlink to your .signature file, e.g. ln -sf ~/.signature ~/.evolution/signatures/signature-5

4) Go to the Mail Account preferences and set symlink .sig as the signature for each account on the Identity tab.

5) No Evolution restart required (small mercies, folks)

[Uche Ogbuji]

via Copia

Ubuntu across the board

My experiment with Ubuntu on two other computers have gone very well, so I bit the bullet over the New Year's holiday (while recuperating from my intercontinental trip) and migrated to Ubuntu on my right arm machine, my Dell Inspiron 8600.

Setting up the laptop proved trickier than the desktops, but I don't think any of my troubles were Ubuntu's fault. The first pass seemed to go well enough. It was really cool when the greeting chime came on too loud, I reached on a lark for the Dell's special software volume buttons (which had never worked under Fedora), and I saw a little volume widget dutifully cranking down the volume for me. That's the famous Ubuntu "it just works" experience.

Then the trouble began. I started to restore key files from the full backup I'd made to an external USB disk. All of a sudden I started getting "unable to write" errors. My file system was read-only. dmesg revealed the dreaded:

[4302337.674000] EXT3-fs error (device hda3): ext3_new_block: Allocating block in system zone - block = 3244032
[4302337.674000] Remounting filesystem read-only

I rebooted a few times, but the file system always found a serious error and remounted read-only after a while. I'm not alone in this. I decided a reinstall was in order. I started the reinstall, but now I started to have a problem where the install disk would boot, but then the installer would claim it could not mount the CD. Again I'm not alone (1), (2). I followed the hints that this might mean a bad install CD and burned a new one. Success this time. It occurred to me that a corrupt CD might have been the reason for the file system problems before, as well.

I resumed the backup, but I now found that the backup tar/bz2 on the USB disk was corrupted. I suspect this might have been related to the file system blow up, because I'd sanity checked the backup before nuking my laptop drive. Oh well, luckily I had a secondary backup of my most important files (my home directory) via rsync to another computer. I restored from that. My Wifi connection kept falling over as I copied Gigs of files via scp, but I got through it. I think this is just persistence of my old Intel Centrino kernel driver problems.

This time I left the install of things such as multimedia and Firefox 1.5 to Automatix, which rocks, and is one of Ubuntu's secret weapons. I've learned that it's much more predictable to install things from Automatix a small group at a time, rather than clicking every checkbox of interest and doing a mass install. The problem with this is that the checkboxes of what's being done by Automtix are not maintained between sessions. You can keep track of what you've done by checking ~/automatix.log.

We'll just have to see over time whether I ever even cast a glance back towards the Red Hat/Fedora family. So far, not a bit of that. Ubuntu impresses my socks off, and I'm glad I've made the switch.

[Uche Ogbuji]

via Copia

CVS log since tag?

My usual trick for creating a "What's changed" summary in my projects is to check CVS for commits since the previous release. SO if the previous release was 24 October 2005 I run

cvs log -NSd ">2005/10/24"

It would be nice if I could do the same thing while specifying the last revision, rather than a date. I wish I could do:

cvs log -NSr<last-rev>::HEAD

but that seems to work only for numerical revisions rather than tags. Does anyone know of any neat hacks to achieve this? Note: if you prefer to advocate Subversion, that's OK, but at least be sure to specify the precise command to do this with SVN so that others can benefit from the example.

Note: this is coming up for me now because I'm wrapping up the packaging for 4Suite 1.0b3 release. One huge new feature: Full DTD support for all the parsers (written in C by the indefatigable Jeremy). One big fix: build support for 64 bit Intel architecture machines.

[Uche Ogbuji]

via Copia

Ubuntu on two out of three

I have three Linux computers that I use to varying degrees. Until recently, they were all the most recent Fedora Core version. I've tried a lot of Linux distros on the different computers—Debian Sid, Gentoo, Mandrake 10, etc. In the past I've always come back to Red Hat or Fedora Core. Honestly, the biggest reason is probably that I know the OS very well. Despite my always going back to Red Hat land, I've never been much of a Red Hat zealot. I don't try to convert people, or anything like that. I don't think any Linux distro is intrinsically superior to any other at this stage. Linux is too advanced, and too mature for fundamental superiority in any of its primary flavors. It all comes down to what fits your head the best.

Unfortunately, although Fedora Core still fits my head best, the community seems to have hit the doldrums, big time. I've always complained about the repository politics, and I always end up going first with the Fedora Extras faction, and then the RPM Forge faction (usually preferring the latter). Since FC4 came out, I've been very frustrated that I cannot get the combination of timely updates and stability that I've come to expect from Fedora Core. I switched from Fedora Extras to RPM Forge hoping for more dynamicism, but I think that was actually a change for the worse this time. At the same time, Ubuntu seems to have breathed a huge amount of energy into the Debian world. I tried Debian once and was very disappointed. Sid broke too often and woody updated way too slowly for my needs as a developer. I tried to learn enough of the system to be able to tame Sid, as Debian Gurus such as Edd Dumbill can, but that's when I discovered that Debian just didn't fit my head. I went back to Fedora. Edd tells me, and many others have corroborated that Ubuntu does an amazing job of dealing with these problems. "Ubuntu just works" is just about an axiom these days. Time to give it a go.

A month or so ago, as an experiment I installed Ubuntu on my third string computer. Besides the installer (which sucks) I was quite impressed with the Distro. Impressed enough that I'm installing Ubuntu on the second string computer as I type. That will bring me much more close and personal with Ubuntu: that's my main media computer: video, music, graphics, games, literary writing, etc. If I'm happy with that the last bastion of FC4 for me is my laptop—my very right arm. It's my main development and communications computer (programming, tech writing, e-mail, Weblogging, IRC, IM, etc.)

My first impression of Ubuntu is that the installer is laughably ummm...retro. I've been installing Linuxen since Red Hat 4.2, and I've see it all, from having to duck into the spare console during an install for workaround hackery to the steeplechase of the Gentoo installer (I swear they're deliberately placing a hurdle in front of users: if you can't complete the scavenger hunt of the install process, they don't want you as a Gentoo user). Ubuntu's installer might be the ugliest and least intuitive I've ever seen. Fedora Core and Mandrake have really nice installers, and honestly, I'm a bit spoiled in that department.

Once I got past the installer the default look of Ubuntu is a combo of really sharp fonts with a theme that's drab almost to the point of depressing. Who chose that brown/copper background and the dun theme? That got changed with the quickness. It's interesting that you only create a regular user, and never set a root password. On my first Ubuntu machine, I couldn't resist switching to single user mode and setting a root password. This second time I'll just leave that alone and see how it goes. Ubuntu set me all up in sudo, so I'll probably be fine for a good while.

I started with the default package manager and added a whole bunch of packages I needed, or wanted to play with. The package management experience is better than Debian's which was already better than Fedora's, but then again, that has always been Debian's well-known strength. I did run into a problem with non-existent "Multiverse" repository option provided in the package manager. For some reason I was not able to install, say Real from the default package manager. It said I needed to enable "Multiverse" but I was having problems with that repo entry. I tried Synaptic and it installed the problem packages with no problem. Synaptic is a real treat.

So far I'm definitely happy with the switch. Many more things just work than in FC4. I do have to look into a few things, for example:

  • Enabling mp3 play in Rhythmbox (I knew how to do this in FC)
  • Figuring out how to loosen the firewall chokehold and how to enable services. Wanting to print and SSH to the machine, I tried the various applets in System->Admin and Apps->System Tools and I can't figure out the equivalent of FC's "Security Level" and "Services" applets.

I'm confident I'll find answers because the Ubuntu community seems as full of useful info as the Fedora community used to be. I already addressed a couple of needs with some easily found resources:

[Uche Ogbuji]

via Copia