Finally, I have released DAViCal 0.9.5. Hopefully this will resolve the series of installation- and upgrade- related problems which plagued the 0.9.4 release.

Thanks for everyone being patient while this release was thoroughly tested through five pre-release versions, and especially thanks to those patient people who helped test those pre-releases.

Now if I don't get too distracted by:

• processing my GPS tracks of the last two weeks around the South Island
• Uploading my last six months worth of photos
• Making sure the kids have done their homework
• Describing the places near where I live.
• ...anything else that catches my interest

... then maybe I will be able to really concentrate on nailing the scheduling extensions work over the next couple of months...

Wish me luck!

The IPv6 wave progresses apace. Well, perhaps not 'apace', but it is moving...

The latest kernel exploit has incidentally had some local fallout in causing more of our boxen here to be upgraded to kernels with IPv6 support, and as a consequence our mail server is now reachable on IPv6. Some have suggested that making it only reachable on IPv6 is a good solution to spam but I suspect that there are still a few mailservers out there that we do want to receive e-mail from which are not IPv6 capable yet!

Since I can now SMTP and IMAP happily over IPv6 I decided it was time to get more adventurous. IPv6 is now in Squid3 head, so I built Etch packages of that and it seems to be 'basically working' in a few places now. We've been using ircd-ircu for a long time for an IRC daemon and it similarly seems that now has IPv6 support, so I backported that to Etch as well.

Packages are available for i386 and amd64 from my repository:

deb http://debian.mcmillan.net.nz/debian etch ipv6
deb-src http://debian.mcmillan.net.nz/debian etch ipv6

If I think of more Etch things that I need for IPv6 I'll put them there too. I do have dircproxy for Etch with support for connecting to IPv6 ircd but I seem to have misplaced the packages somewhere. If you're keen on seeing that then I'm sure I can reconstruct them somehow...

Now that we are having increasing amounts of IPv6 around some things are starting to reduce down to a 'Small Matter of Firewalling', which is suggesting to me that we will need manage our firewall rulesets differently for IPv6 than we have for IPv4.

In a lot of cases we can turn on/off large chunks of access related to a particular person/organisation by disabling a VPN, with the firewalling being a somewhat static monolithic overriding control above that. With the control potentially moving away from the VPN, and more directly into the firewall rules, we will need clearer association mechanisms in place. Of course we will continue to have VPNs, but they might become somewhat simpler, reducing in many cases to encrypted tunnels between exact endpoints.

A GPS is one of those toys that I have wanted for a very long time. So last year I finally marshalled enough excuses in one place, lined them up and plunked down some money for a Garmin GPSmap 60CSx in the vague belief that there's enough Linux software out there which understands GPS, and the Garmin is a brand that seems to have mature Linux support. And besides, I'd been told it was a good model that would do what I wanted and then some.

Once I got it I naturally wanted to do stuff with it, and in particular I wanted to connect it up to my laptop which (of course) is running Linux and I found that it wasn't nearly as trivial as I had thought.

This prompted me to run around finding software to use with it, so here's my capsule review of my journey from "extremely naive" to "very naive". Perhaps I'll also learn something from the comments of people who are further down the track.

My first resort was "apt-cache search gps", of course, which immediately brings up such fine sounding programs as "gpsman", described as "A GPS manager" (I haven't managed to get it to do anything yet), "gpstrans" which since it specifically mentions Garmin GPS sounded like just the ticket (it appears to be quite old and superseded). There was some useful stuff as well.

Getting Maps on the GPS

Since the Garmin GPSMap series will display maps, I wanted to be able to get some mapping data on there. There are two map sources I was interested in:

• Map data from NZ Open GPS Maps which gives comprehensive coverage of New Zealand, and is free and community maintained, albeit with some annoying uncertainties about the licensing of the data, and it's being tied to Garmin GPSs.
• Map data from OpenStreetMap.org which gives excellent coverage of many areas of the world, and is free for use under a creative commons license.

Using the NZ Open GPS Maps

The NZ Open GPS Maps are built specifically for the Garmin GPS, so getting them onto my GPS was relatively simple once I found out the exact command line was remarkably hard.

The program that I needed to find to be able to get the maps onto my GPS was sendmap20 which is a free (as in beer) download from the cGPSMapper Website.

I downloaded the maps from the NZ Open GPS Maps hosted on the cGPSMapper site. I needed the files identified as 'binary' files - the installer is a Windows program and no use on Linux. They all have filenames like 64000012.img.

The command-line I first used to install the maps onto the GPS was:

sendmap20 -t/dev/ttyUSB0 640000??.img


That works fine, but if you have even a moderately sized micro SD card in the GPS there are much faster ways. As I found more free maps I found much better ways to do it.

An alternative approach

In the GPS (well, in my one anyway) there is a micro-SD card which is FAT formatted and all of the installed maps are in one humungous combined image of all of the uploaded maps. This means that there is no way to straightforwardly add/remove maps without creating that image, and re-uploading the whole thing. The image file is called 'gmapsupp.img' and is in the 'garmin' subdirectory on the SD card.

This means that you can create the file (or even several different files, with different maps on them) and move them directly on there much quicker, either by taking the SD card out and using it in a USB2 reader (for really big files) or switching the GPS to operate in USB Storage mode (which is USB1, but OK for smaller files).

You can create the IMG file to copy in as gmapsupp.img with sendmap20 also, with a command-line like:

sendmap20 -lNZOpenGPSMaps.img 640000??.img

and then, if you mount the GPS as /media/GPS, you can copy that file on directly

cp NZOpenGPSMaps.img /media/GPS/garmin/gmapsupp.img


QLandKarte

Another program I found useful specifically for dealing with the Garmin GPS Map format is QLandKarte, which understands the native Garmin format and will display the maps in a graphical window.

You can also point and click to select maps, building up a specific set that you can then upload to the device from within QLandKarte itself. This seems to operate a lot faster than sendmap20, presumably because it's driving the USB directly, rather than through usb_serial. To get it to work I seem to have to (a) sudo rmmod garmin_gps and (b) sudo QLandKarte, however, which is definitely not ideal.

For the moment I will continue to use sendmap20 to create files which are sets of the maps I want, and will put them on the device in USB storage mode.

QLandKarte is still very useful, however, for looking at maps and deciding if they are worth bothering with. I used it in this way to select the Australian maps I took with me when I went to Linux.Conf.AU recently. There are more free Garmin GPS maps.

OpenStreetMap.org

Those Australian maps came from OpenStreetMap.org and ultimately I expect that it will become the best source of data for creating maps for my GPS. OpenStreetMap.org is an attempt to provide community-maintained maps for the whole world and which seems to have made significant progress in UK, Europe, America and Australia as well as many other parts of the world.

The data from OpenStreetMap.org does not have the same licensing restrictions as are present even in the NZ Open GPS Maps data (which is relatively free, but has problems similar to the old-style BSD license).

There are also some good mapping interfaces to the OpenStreetMap data although the searching currently still leaves something to be desired. Interfaces are also available for extracting subsets of the data, which is in a fairly straightforward XML format, or you can download the whole lot, but at nearly 3GB for the bzip2 compressed version you won't do it every day.

I gave a brief overview talk about OpenStreetMap.org while I was at Linux.Conf.AU and it seemed to go down quite well (I gave it several times, in fact). You don't need a GPS to contribute to the project, and of course the maps which are being created are usable for many purposes beyond their usefulness on GPS. Some people apparently even print them out, but that just seems weird!

LCA officially started today, with a great opening keynote by Bruce Schneier very nicely pointing out the differences between feeling secure and being secure, and with a few good examples of where these are mismatched and action needs to be taken to fix the mismatch. Good stuff - catch it on video if you get the chance.

Of course for me LCA started on Monday when I got roped into presenting a lightning talk at the Community Wireless miniconf on OpenStreetMap.org which is something I've been fiddling with recently, and a project which definitely deserves more attention. Since nobody else turned up to give their lightning talks my 3 minute presentation was stretched severely out of shape, but the Community Wireless miniconf was generally pretty interesting and I stayed there all day in the end.

Yesterday was the SysAdmin miniconf, and as one of the organisers there was no chance for me to look around at what else was going on. It was a full-on day, and the feedback I have from everyone is that it was a good miniconf. Certainly it was good enough to attract more people than could fit in the 120 seat theatre we initially had allocated so that we had to move into a much larger theatre. A big thank-you to the organisers for so quickly recognising and reshuffling things so that was possible.

All signs so far definitely indicate another great linux.conf.au

A curiously interesting thread has sprung up on the Xorg mailing list recently regarding the lack of bandwidth the developers might have for accepting patches. Bernardo Innocenti, who hacks on OLPC stuff amongst other things, wants to get some patches reviewed for acceptance, but there aren't enough people around with the time and inclination to review and accept his work.

He did a quick and dirty analysis of Xorg LoC vs Linux LoC, to give a "X codebase is roughly 1/2 Linux codebase" (which surprised me, actually, I'd always thought it was larger) and then a similarly rough analysis that suggests that Xorg has roughly 1/20th of the developer base.

To some extent Bernardo's metrics are arguable, but the basic facts are that Linux gets the lions share of the developers. Is this what we should expect? Is this a healthy way for the free and open source software community to support the development of what I personally would like to see as the future desktop base?

Linux does a fantastic job as an OS, right now. It's been doing it for years, really, but we still need to get behind some of the differential further up the stack. And all credit to those stalwarts of X development who are shouldering more than their fair share of the burden.

So if you were looking for an open source software project where you can be appreciated, don't overlook X.

Quite a few people seem impressed with the new release of Leopard, and are now looking for a CalDAV server to use with their shiny new iCal app. Unfortunately it seems that Apple wrote this primarily to work with their own (free, open-source) calendar server, which has the side effect that it doesn't work with DAViCal.

Part of the "doesn't work" is due to DAViCal not implementing some areas of the CalDAV specification, which is fair enough. Part of it is due to DAViCal not implementing some draft extensions to the CalDAV specification, which I can also understand, since it allows them to provide some useful features that those extensions are designed to support. There also seem to be some parts of the "doesn't work" which are due to a dependence on extensions beyond either of these cases, which is a little more disappointing - and quite a bit harder to implement.

So far I have made some fixes to the first point, and some additions towards parts of the second, but as of today it still does not work. This is complicated by my not having access to a Mac. Things are looking up, however, because Tom Robinson has kindly agreed to loan me a Mac running Leopard from next week.

In order to "clear the slate" for that, I will be releasing a 0.9.2 over the weekend with the various minor enhancements and fixes that have been applied over the last week. So although this upcoming release will let you add your DAViCal account to iCal 3, it still won't actually work with it. I'm hoping that ready access to the application will enable me to correct that fairly quickly.

Also waiting in the wings (and which unfortunately won't be in 0.9.2 either) Maxime Delorme has been working on SyncML support, and is nearly ready with a patch, so we can look forward to that addition fairly soon also.

I hate spam!

Which probably puts me in the same camp as 99.99999% of the world. The other 1 in 10 million are, of course, the spammers, who seem to take the space invaders approach to sending e-mail: we'll keep sending you more until you die.

A few years ago I used to only receive perhaps 1 every 100 seconds, which was pretty annoying, but Spamassassin was quite able to filter out 99% of those and let through about 1-2 each day, which I could deal with. My spam levels increased to maybe 1 every 20 seconds, and late in 2005 I implemented a second layer of spam filtering on my laptop using DSpam. This worked quite effectively, but DSpam is really not the tool for the job - it's much more appropriate as a company-wide antispam solution, and potentially as a replacement for Spamassassin. It drove me nuts on my laptop because it's resource usage slowed down the interactive response.

When I got my new laptop at the beginning of the year I decided against continuing with my rather baroque mail setup and to leave the spam filtering on the server. What I didn't realise is that my spam rate had increased again to around 1 every 8 seconds, and it has been slowly driving me to distraction ever since. It seems to have cranked up another notch recently, to perhaps 1 every 3 seconds now, so that 1% making its way through Spamassassin was getting to a very annoying several hundred each day. The longer I took to resolve it, the more time I would be wasting dealing with it every day.

What I chose to apply on this occasion was CRM114, which I had some vague idea might be able to help. I was fairly impressed by the relatively simple install, but what completely blew me away was the speed with which it was able to learn to be useful. Starting from scratch, it seems to be correctly classifying over 90% of my incoming mail after about 12 hours of training, on a total of only 75 'Unsure' messages. Even after only an hour it was getting over 50% (I'll describe my actual CRM114 installation process in a comment below). So far there have been no false positives.

Now that CRM114 is installed I will be able to look into some of it's other mail classifying features too, and I'm really looking forward to that too.

Last week I installed Ubuntu Gutsy onto Heather's laptop. While Gutsy seems to be an easy task for most situations, installing it onto a Pentium 366 laptop with 200M of RAM and (particularly) an 800x600 screen was harder than it perhaps should have been.

I'm sure that most installations these days aren't 800x600, but the graphical installer in Gutsy seems determined to make this painful. I had to move the toolbars to the sides of the screen, and then I could see the top half of the buttons on each page. It was like the page was sized for 600 vertical pixels, but the designer had forgotten about toolbars and title bars - not that I could see any screens in the process I followed that needed more than 5/6 of that screen anyway. Eventually I got it installed, and it even seemed to run OK once we booted into it. That's "OK for a 200M P366 with an 800x600 screen" though.

Looking around at the price of a new laptop made putting up with that sort of performance a whole lot less palatable. The Acer Aspire 5310 (with free RAM upgrade) was $898 at Dick Smith, with a $99 cashback offer. A quick google shows that it's using the Broadcom 43xx wireless which isn't even close to being the best, but can be made to work with Linux. Everything else seemed likely to work, so we bought it.

Installing Gutsy on it was nearly trivial, though I had to install bcm43xx-fwcutter on a different PC (my laptop, which is running Debian, in fact) to get the firmware for the WLAN before I could get the wireless working. I'm surprised that Broadcom still don't make that firmware publicly available somewhere, rather than forcing people to jump through the sort of hoops that would get them wanting an Intel chipset next time.

Anyway, everything installed very easily, and the laptop is working quite nicely. Strangely neither sound, nor suspend to ram are working out of the box. They're not so important in this case fortunately, but perhaps in due course I'll try and get them working and post some details about it.

Much harder has been getting the fabled 'cashback' from Acer. I think I now know what I'm being paid $99 for. Firstly the only way to get your cashback is by registering through a webpage. Heather's first attempt to do this resulted in an error from our proxy about a malformed request, so I got called in. I tried registering using on my laptop, but couldn't even get to the cashback page. I then tried using IE6, with similar results. So perhaps it's my PC? I tried using a different PC, with the same result again!

We tried ringing them up, but they were absolutely determined that (even after 20 minutes on the phone) they were not going to accept that information over the phone. So the only way to get the cashback from Acer was via their thoroughly broken website. Even their Contact Acer page is broken in firefox just showing a blank. Firefox users need not apply.

Eventually, while spending some time in front of Heather's main computer (which had made it all the way through to submitting their on-line form before failing) I realised that the error she was getting was a proxy error from some in-form javascript submitting an invalid request, so I disabled the proxy, the form finally worked, and I managed to apply for the cashback. Now we just have to send the printed form in, along with some blood from our firstborn, the ashes of my grandmother, various barcodes, receipts and toenail clippings and we're sweet. They say they'll send us some money within 30 days. I think we should maybe frame it or something. I just know I'm going to feel really inclined to take advantage of cashback offers in future.

In Other News: DVD Slideshow

Meanwhile I've been playing with DVD Slideshow which seems to be just what my parents have been after for a while, so they don't have to keep their favourite photos on the camera to be able to show them off on someone's TV. It's great! At least it is great now after I changed all the calls to ffmpeg to add a 'k' after the bitrate parameter. But that's Open Source Software, I guess. I'll send a patch to them... :-)

I have not been able to put a lot of effort into DAViCal over the last couple of months, since my father was diagnosed with stomach cancer in early September, and he died on 2nd October.

So here it is, finally, including a lot of refactoring work around the handling of DAV/CalDAV REPORT requests and implementation of the DAV::principal-property-search report. This also requires an upgrade to the latest AWL library (0.20), which includes a complete rewrite to the class used for parsing and rendering iCalendar data.

This release is recommended, since you will need some of this stuff to support the upcoming Mozilla Calendar 0.7 release properly.

At this point I have only released the files to http://debian.mcmillan.net.nz/ and I'll push it out to a wider audience if I don't here screams of anguish from people in the next few days :-)

This release does not have any associated database changes, so it should be a simple matter to install the upgraded code.

This week has been a week of pictures for me. I was looking around on Wikipedia earlier in the week and decided that the articles around the Porirua area where I live was looking a little barren. So I've been out taking a few snaps to liven it up a little. The articles themselves don't look that great either so I'll see if I can't put some effort into improving them as well.

As a result though, I've been playing with photographs and I'm finding that the graphics landscape on Linux has improved significantly since last time I was playing around with things. I've been using Hugin for making panoramas since someone pointed me to it at Debconf5 in Helsinki, but version 0.7beta4 with Panorama Tools 13 is an immense improvement. It seems that now if I take my photos the right way I can expect Hugin to create a panorama with only a couple of minutes of time, and no real effort to speak of. I built some Debian packages of Hugin 0.7beta4 and libpano13 which are here, if you're interested. Sadly, it seems that the current maintainer has not updated these packages for a long time.

Something I've also been trying to do properly for some time is to produce decent HDR images. Although the tools have been around for a while they are somewhat inaccessible to people, taking a long time to gain any kind of intuitive understanding about how they work and what parameters a person should use. The answer, it turns out, is something like the Hugin one: provide a GUI front end. Well there now is one, and it's damn good. I've now started taking a few photos in triplicate so I can fiddle around with QtPfsGUI and learn to recognise the situations where HDR can make a better photo. It's not an addictive name, but it's great software and when I've managed to get my head around it I will create an HDR area on my gallery and put a few fun images in there. I've built some Debian packages of this also, but I've never had to deal with something using QMake before and it doesn't seem particularly friendly to packaging. Probably I'm just beating on it the wrong way. Anyway, here are some Debian QtPfsGui packages for anyone who is interested. Maybe if you know how better to drive qmake from a distribution point of view you could send me some tips.

Something that I really like about both Hugin and QtPfsGui is the way that they are providing a GUI framework for some pre-existing command-line tools. The separation of UI from function is a classic application of the Unix model that will be familiar to anyone who has ever piped find into xargs, and it really does allow for the whole to be greater than the sum of the parts. What is interesting about the two above, seems to be that by providing the UI they have managed to breathe some new life into the underlying functions as well.

The next thing that I'm really looking forward to is the upcoming 0.46 release of Inkscape SVG editor. This totally awesome program just gets better and better and a bunch of the upcoming enhancements come from the Google Summer of Code. Actually all of these programs are quite a bit better recently because of the sponsorship they have received from the GSoC.

There have been a few Catalyst people mentoring some GSoC projects - mostly around Moodle, Lisp and Git - so a couple of the Catalyst folks will be off to the big meetup / review in a couple of weeks. I hope those lucky stiffs pass on my thanks to everyone involved.