On reading the 2006 QuickStats National Highlights from the Statistics New Zealand website, I am prompted to write...

Dear Statistics New Zealand,

In the "2006 QuickStats, National Highlights" document available from here:

http://www.stats.govt.nz/census/2006-census-data/national-highlights/

On the final page is a link to the Excel viewer which would supposedly enable me to open Excel files.

Firstly, the link does not seem to work. When I click on it my browser opens at the location:
http://wipe04/NR/exeres/727CCC9A-22D0-4E71-960B-D72260540535.htm?NRMODE=Unpublished&WBCMODE=PresentationUnpublishedPreview#excel


Which my web browser (correctly) assures me is not a valid domain name. You might want to correct that.

Secondly, the "Excel viewer" is only a solution for those people using Microsoft Windows or Apple Mac OS X. People using other operating systems such as Linux will not be able to install or run the "Excel viewer" application.

A better solution would be to provide a link to OpenOffice.org which provides a fully functional spreadsheet program that is capable of reading such files, and which is:

• freely available
• freely redistributable, and
• will work on a much wider variety of operating systems

Many thanks,
Andrew McMillan.

Actually I remember filling in that Census form online, about a week before the census, leaving out the questions that related to the night itself. Then on the night I discovered that I had already done it, and could not go back and add the answers to those questions.... Oops!

And when I try and post this as feedback to the stats website, I find that:

Server Error in '/CmsApp' Application.
Runtime Error
Description: An application error occurred on the server. The current custom error settings for this application prevent the details of the application error from being viewed remotely (for security reasons). It could, however, be viewed by browsers running on the local server machine.

Details: To enable the details of this specific error message to be viewable on remote machines, please create a <customErrors> tag within a "web.config" configuration file located in the root directory of the current web application. This <customErrors> tag should then have its "mode" attribute set to "Off".

<!-- Web.Config Configuration File -->

<configuration>
<system.web>
<customErrors mode="Off"/>
</system.web>
</configuration>

Notes: The current error page you are seeing can be replaced by a custom error page by modifying the "defaultRedirect" attribute of the application's <customErrors> configuration tag to point to a custom error page URL.

<!-- Web.Config Configuration File -->

<configuration>
<system.web>
<customErrors mode="RemoteOnly" defaultRedirect="mycustompage.htm"/>
</system.web>
</configuration>

Man, I sure wish I could change that file and help them out with that little problem. I know: I'll send them some feedback, telling them! Yeah, that'll help...

At Catalyst, when there were only 15 or so of us in the office, we used to have a running joke about clients saying "I would have thought", where we would use that phrase to introduce progressively more ludicrous requirements, because we had the occasional client who used that phrase to get work done which had not been written into the contract.

"I would have thought that a system for finding computers on an international network would give some thought to that majority of the planet who don't use the english alphabet.¹

Yeah, sure. That'd be simple, right?

Of course such a phrase really indicates that whether they did (or did not) actually think of that requirement at the time, they certainly never documented it. Either they did think it, and simply assumed that everyone knew their business like they did (and we've all been guilty of making that assumption). Or they didn't.

The joke got to a point where we occasionally found it difficult to contain our giggles whenever a client actually came out with it, and for me at least that continues to this day.

As 'in' jokes go, it was an extremely valuable one to have around though, because it made us notice those points where clients were wandering away from the specification, giving us a decision point where we could clearly either acknowledge our failure to identify the requirement, or to argue whether such an assumption should have been reasonable for us to have understood, and so forth.

I'm reminded of this today, when I read RFC2445, in particular it's definition of PRIVATE vs CONFIDENTIAL.

The document does not give a lot of clues, and seems to trust in the meaning of those words to define which is stronger.

My initial inclination was to go with the ordering of the terms: PUBLIC => PRIVATE => CONFIDENTIAL. That, I thought, seems reasonable. In all the movies the nearly secret stuff is all stamped "CONFIDENTIAL", and then stuff gets stamped "TOP SECRET", and in such movies I'm sure that "PRIVATE" is reserved for signs on doors, rather than on secret files.

I immediately had to reconsider, however, when someone else from the other side of the world (literally) decided that PRIVATE should be the secret stuff, and CONFIDENTIAL was merely "somewhat secret". A small bit of googling suggested that I could well be in the minority on this one, so I will adjust my worldview, correct my software, and possibly even stop watching such silly movies.

Still, I thought it would be nice to deliver these conclusions in the place where they belong: right there on RFC2445. Sure, I could write to the authors and berate them for their wishy-washy language:

"... The access classification of an individual iCalendar component is useful when measured along with the other security components of a calendar system (e.g., calendar user authentication, authorization, access rights, access role, etc.). Hence, the semantics of the individual access classifications cannot be completely defined by this memo alone. Additionally, due to the "blind" nature of most exchange processes using this memo, these access classifications cannot serve as an enforcement statement for a system receiving an iCalendar object. Rather, they provide a method for capturing the intention of the calendar owner for the access to the calendar component."

Oh how very fucking useful. This is supposed to be a specification! The intention of my friend from France when he says "PRIVATE" clearly differs from my own, but that can't easily be arbitrated without a frame of reference which the standard should, in theory, provide.

After I calmed down a bit I was reminded of how some software that I use allows people to annotate the manuals, clarifying things, providing examples and so forth, which I have found incredibly useful from time to time, and it made me wonder whether a site that allowed people to generally annotate RFCs (or other documents) could also be useful.

So: do you think it would it be useful to have some room for interpretation?

¹ That one is not a quote, by the way - I have a very good idea of exactly how much debate has gone on around that, and how stupidly complex the issue has become.

I just downloaded a trial version of one of the latest version of the Progress OpenEdge database because of one of those applications I wrote in different days, and which I am unfortunately stuck maintaining.

<RANT>

I find it astonishing that a database which has been operating on a Unix environment for close to twenty years will install itself in a directory tree where:

• Every directory is mode 777
• Every single file is marked as executable

I am nearly speechless. No wonder their market share has sunk so low that they can only claim 60,000 customers world wide.

As databases go Progress is just fine, of course, and we use it for one notably mission critical application in particular (though probably not as significant as some of our larger PostgreSQL based applications).

Just make sure that you "chmod -R go-w ." in the directory you installed it into as soon as possible...

</RANT>

Meanwhile, I guess I'm putting off figuring out whether the newer version will work OK with my 'favourite' 10 year old application, so I better get back to it.

Grant reminded me today about the sad story of Craig Shergold (or here, of course).

It must be such a pain to be the target of this sort of thing, and yet it continues. I recently received a sad story about someone supposedly suffering in Auckland, but how am I to know if it is true? If I even enquire about it's truth I may be contributing to the problem!

I'm sure that many people receive more of these than me, perhaps because I have historically come down pretty hard on people near and dear to me who have forwarded me these sorts of things. Over the last ten years my approach has softened somewhat: I have switched to forwarding them, by return e-mail, the following warning (I have no idea where I got it from - I certainly didn't write it):

*********************************************************
WARNING, CAUTION, DANGER, AND BEWARE!
Gullibility Virus Spreading over the Internet!
*********************************************************

WASHINGTON, D.C.--The Institute for the Investigation of Irregular Internet
Phenomena announced today that many Internet users are becoming infected by
a new virus that causes them to believe without question every groundless
story, legend, and dire warning that shows up in their inbox or on their
browser. The Gullibility Virus, as it is called, apparently makes people
believe and forward copies of silly hoaxes relating to cookie recipes, email
viruses, taxes on modems, and get-rich-quick schemes.

"These are not just readers of tabloids or people who buy lottery tickets
based on fortune cookie numbers," a spokesman said. "Most are otherwise normal
people, who would laugh at the same stories if told to them by a stranger
on a streetcorner." However, once these same people become infected with the
Gullibility Virus, they believe anything they read on the Internet.

"My immunity to tall tales and bizarre claims is all gone," reported one
weeping victim. "I believe every warning message and sick child story my
friends forward to me, even though most of the messages are anonymous."

Another victim, now in remission, added, "When I first heard about "the
sulfnbk.exe virus" and Good Times, I just accepted it without question. After
all, there were dozens of other recipients on the mail header, so I thought
the virus must be true." It was a long time, the victim said, before she
could stand up at a Hoaxees Anonymous meeting and state, "My name is Jane,
and I've been hoaxed." Now, however, she is spreading the word. "Challenge
and check whatever you read," she says.

Internet users are urged to examine themselves for symptoms of the virus,
which include the following:

The willingness to believe improbable stories without thinking the urge to
forward multiple copies of such stories to others a lack of desire to take
three minutes to check to see if a story is true T. C. is an example of someone
recently infected. He told one reporter, "I read on the Net that the major
ingredient in almost all shampoos makes your hair fall out, so I've stopped
using shampoo." When told about the Gullibility Virus, T. C. said he would
stop reading email, so that he would not become infected.

Anyone with symptoms like these is urged to seek help immediately. Experts
recommend that at the first feelings of gullibility, Internet users rush to
their favorite search engine and look up the item tempting them to thoughtless
credence. Most hoaxes, legends, and tall tales have been widely discussed
and exposed by the Internet community. Courses in critical thinking are also
widely available, and there is online help from many sources, including:

Department of Energy Computer Incident Advisory Capability at

http://www.ciac.org/ciac/

Symantec/Anti Virus Research Center at

http://www.symantec.com/avcenter/index.html

McAfee Associates Virus Hoax List

http://us.mcafee.com/virusInfo/default.asp?id=hoaxes

Urban Legends Reference Pages

http://www.snopes.com/

Wikipedia on Urban Legends

http://en.wikipedia.org/wiki/Urban_legend

Datafellows Hoax Warnings

http://www.f-secure.com/virus-info/hoax/

Those people who are still symptom free can help inoculate themselves against
the Gullibility Virus by reading some good material on sources, such as:

Evaluating Internet Research Sources

http://www.virtualsalt.com/evalu8it.htm

Evaluation of Information Sources

http://www.vuw.ac.nz/~agsmith/evaln/evaln.htm

Lastly, as a public service, Internet users can help stamp out the Gullibility
Virus by sending copies of this message to anyone who forwards them a hoax.

(line of asterisks here)

This message is so important, we're sending it anonymously! Forward it to all
your friends right away! Don't think about it! This is not a chain letter! This
story is true! Don't check it out! This story is so timely, there is no date
on it! This story is so important, we're using lots of exclamation points! For
every message you forward to some unsuspecting person, the Home for the Hopelessly
Gullible will donate ten cents to itself. (If you wonder how the Home will know
you are forwarding these messages all over creation, you're obviously thinking
too much.)

*********************************************************
ACT NOW! DON'T DELAY! LIMITED TIME! NOT SOLD IN ANY STORE!
*********************************************************

A wee bit dated now, perhaps, but as Pete Bulmer said when I pointed that out to him "Are you kidding? That stuff circles the internet for years. People don't get any smarter ... and if it is dated, that just makes it more poignant."

Indeed. So I've dusted it off, cleaned up some of the links, and pasted it up here where more people will be able to refer to it - and in the hope that you can retaliate with it next time someone tries to infect you with the gullibility virus.

I've been trying for a long time to find something that will match my music needs, but I think I give up, so I have finally decided to release my "Adorno" music server out into the big blue room.

Some people suggested Amarok, but while it does have some web interface plugins they really are Teh Suck(tm) for all-the-time use. Using Amarok over an SSH connection seems to soak network bandwidth to the max, as well, and is quite sluggish. Amarok is nice, but I don't play my music on the computer in front of me.

I looked at SlimServer, but it requires me to run some Java doohickey on the music player which (surprise, surprise) expects the server to have a GUI.

I looked at MPD as well. This is probably the closest thing to something I could use, and if I hadn't already written my own music server years ago I would probably go with this. As it is I have something which kind of works, and which has records of the last 20,000 odd tracks I have played. It only really has a few bugs, which are minor enough to have not been fixed for a couple of years, so I should just get stuck in and do the work.

What crystallised this chain of thoughts for me was yesterday, when everyone got a wee bit excited about the Catalyst IT group on last.fm I decided to finally sign up for an account on there. Of course if I had an account, I had to have some way of actually putting my playlist on there...

... 10 minutes later I had managed to find the Audio::Scrobbler library, and an hour later I had hacked the support for it into Adorno. Looking through the code of the music daemon component of Adorno I realise it ain't that bad. There are a few improvements it could do with, but the basic approach works just fine.

So I'm now listening to Charles Mingus, and it's all being scrobbled up to (by?) last.fm, although I guess strictly this isn't all going to be my choice of music. Under pressure from one of the shorter members of the household I have been known to play "The Wiggles", "Buzz O Bumble" and other stuff I am heartily sick of, but which he continues to adore.

So now I had firmly committed to releasing my code, I had to get rid of the revision control on there (I was using Darcs, since that's the project I was starting when I wanted to try Darcs out) and replace it with Git. I don't think there is any value in retaining all my old history so I just moved the _darcs directory out of the way and "cg-init" in the root of the project.

Then back on my laptop, I just clone from the remote project root and start editing away. Eventually I decide to "cg-push" and my changes go back into the project root on the music server. Except, of course they don't: they go back into the .git directory in there and when I ssh in and run cg-status in the project root it wants to undo all my good works! I think that's a bad Andrew for even thinking of doing something so stupid! So I'm now using "cg-reset" to put my code into operation on my music server. Ouch!

Oh well, I guess that means that Debian packaging must be early on the list :-)

This week I had to visit Queenstown to do some work for a client. Queenstown is surrounded by beautiful landscape, and I had great weather for the flight from Christchurch to Queenstown, including being able to very clearly see Mt Cook, Lake Hawea, Wanaka, Mt Aspiring, and the Crown Range from my window seat. So it was extremely frustrating that my camera batteries only lasted until Christchurch.

When I got to Queenstown I realised that I had also forgotten my sunglasses, but I didn't need them for long as I was soon back to wrangling the computer programs into the right shape. All was "mostly working" at the end of the first day, which was the kind of progress I had hoped for so I celebrated with a very nice dinner at Tatler which was bizarrely nearly deserted. Such a restaurant in Wellington would have been packed any night of the week, I feel sure. I wandered around Queenstown after dinner wondering how they manage to retain that small central core for something which is so incredibly touristy. Touristy without being tacky. Well, not too tacky anyway. It's quite impressive that they have managed to hide away all the industrial areas somewhere that I didn't see.

The next two days the weather was increasingly wet with two very nice breakfasts at Joe's Garage to start each day off. I didn't mind the weather too much as I was inside working ... until the time came to leave on Friday. After a couple of hours wait my first flight was cancelled and I was bumped to a later flight, which was also cancelled so I got moved to one on Saturday afternoon. I've had to cope with travel disruption so infrequently that I was really quite flummoxed for a wee while. I called up one of the people I had been working with, and he was kind enough to invite me for dinner while I figured out what to do (thanks Craig - it was much appreciated).

The next morning dawned. Wet. Again. And I really didn't trust that 1:00pm flight to actually get out. The forecast was for Queenstown to partly clear, and then close in again, but I get the impression that this airport is closed a lot more than most, and from the flight path in I'm not altogether surprised - although it felt like we could touch some of those peaks, I think I would rather not do so, thank you very much, and I'm sure the pilots feel the same way.

So I hired a rental and drove to Christchurch, rebooking my flight en route. Unfortunately I took a wrong turn at Cromwell, I think, and it took a lot longer than it should have (via Dunedin) so I got to Christchurch just barely before my flight left, very flustered.

A long day, but at least I get to sleep in my own bed at the end of it :-)

I just found some blogging by Lisa Dusseault on CalDAV and discovered that CalDAV is now a 'Proposed Standard'. That may not mean much, but remember that IETF eventually ends up calling these things "Request For Comments". Lisa and others have spent considerable time and effort developing their specification to this point and congratulations are definitely in order.

So: what's interesting about CalDAV?

People have been asking me why calendaring, and in particular CalDAV, is such a hot button for me recently. The answer has evolved over time, and really comes in several parts...

Firstly, my desire for this is not recent

Shared calendaring has been a "black hole" in the Open Source universe for way too long. Several employers ago (in the early '90s) when I worked for ${LARGE_GOVT_DEPT} I used to schedule meetings using ${GROUP_CALENDAR} and it was damned useful. Later I moved on to work for a much smaller organisation, but ${GROUP_CALENDAR} there was still extremely useful - just in a very different profile of use.

When we got together and started Catalyst IT in 1997 I started turning my back on the 'Windows' world, looking at Linux as something that was interesting and Open Source as a movement I wanted to be involved in.

In 1998 I switched full-time to using Linux on my laptop, but in doing so I denied large chunks of application space which simply wasn't available. Heck, my desperate search for a decent open source word processor caused me to be cited in a 'why the concept of XML documents should not be patentable' recently since I really used some early versions of AbiWord to write letters to clients, and I still had them on my hard disk!

Since then I have seen the application space on Linux slowly populate with great programs that do large chunks of what I want to be able to do with my computer. I've contributed to that too, in programming small applications, filing bugs, promotionally and in other small ways.

So Why CalDAV?

After a while people started to talk about DAV, and perhaps this would be a way of providing a group calendar application. Unfortunately the way it ended up being used by most of the applications that did support it was to use it as a place to dump a person's entire calendar as a single file, or maybe just their free/busy time.

That ends up being flawed in a whole bunch of ways: how do you delegate responsibilities among people, for example?

Although DAV evolved over time to answer some of those questions, the applications that were paying lip-service to DAV failed to evolve in step.

A Star is Born

In the calendaring firmament, at least! Lisa Dusseault started to produce specifications for calendaring extensions to DAV and then she got a job with OSAF specifically to do that. That standards development process has taken three years, but we're finally near the end of it. Here's the description of CalDAV from their website:

"CalDAV models calendar events as HTTP resources in iCalendar format, and models calendars containing events as WebDAV collections. This allows users to publish and subscribe to calendars, share them collaboratively, synchronize between multiple users and synchronize between multiple devices.

Which is an excellent conceptualisation of what is (was) needed.

Meanwhile I was still looking around for a solution to the group calendaring problem, and not just for myself: Clients were also regularly asking what could be used to provide them with a shared calendar solution, and if they were asking me that then they were looking for an open source answer.

Now we have a standard, where are the implementations?

At this point it kind of becomes "chicken and egg". There are no servers, so people can't easily test the clients they are developing, and there are no clients, so people can't easily test their servers. Well a few people are working on these things, of course. On the client side, Mozilla is adding CalDAV support and CalDAV support snuck into Evolution in late 2005. Cyrus Daboo (one of the other authors of CalDAV) implemented it in his 'Mulberry' free-but-not-yet-open-source application, and there are a bunch more which are associated with client-side projects as well.

Looking around though, it seems that there are no simple servers. Some of the servers that are being written seem likely to fall into the trap of working best with their own client application, or they are simply proprietary solutions. Or they have requirements that are (to me) not particularly desirable, such as being written in Java.

I don't have a problem with Java, per se, but when you might want to run 20 applications on one server it always seems to want to get bloated and to eat administrator time...

Hence the Really Simple CalDAV Store

Eventually I got fed up with looking around the open source landscape every few months, searching for group calendar servers and finding the continuing promise of jam tomorrow, but no jam today.

Around May this year I decided to build my own, and around 4th of September I finally had enough answers to start work on it.

Yesterday I made a significant milestone release, with support now for multiple languages in the administrative interface, delegated permissions are working in a basic manner and the CalDAV operations used in Mozilla Calendar and Evolution are fully supported. Most of the CalDAV operations in Mulberry are supported too, although some of the more arcane ones will be my work for the coming month.

Thanks!

Thanks to Cristina, Lorena and Guillaume for their help in translating the administrative interface. I hope to thank a few more people for more translations soon also!

And also, thanks to Lisa, Cyrus and Bernard and everyone else who was involved in making the CalDAV idea a reality. Great work.

A while ago someone suggested to me that the Peppermill in Porirua was a nice cafe so we went there after our tri-weekly library visit today, and I was kind of disappointed. After paying $2 for a 100ml glass of milk, and a fluffy in a tiny espresso cup, or $8 for about 1/2 a glass of wine they forgot to even deliver Heather's Chai Latte.

And then our "meals" arrived, which very much reminded me of the worst days of Nouvelle Cuisine. Back to Kaizen next time, I think.

Here's a very interesting report from Telecom and Alcatel about ADSL in New Zealand... Kind of surprising to see it publicly available, to be honest.

Days like this I'm glad I'm on the end of a wireless link. And I don't have to put up with the ridiculous latency of ADSL as well.

I live more than 2k from an exchange, and used to dangle off the end of one of those sucky ADSL connections where Heather would wipe out the usefulness of my SSH connection whenever she downloaded something. Every time another customer goes onto that stuff it seems that it gets worse for everyone else in the same cable bundle.

Awesome. And that is not talking about the current plan changes, but about the ones that happened in April.

The Ubuntu theme for art.ubuntu.org doesn't work on Gallery 2.1, so I made another version for gallery.ubuntu.org.nz which does. The plan is to use this for a competition, coming soon...

And if anyone else wants the Ubuntu Art theme for Galley 2.1, here it is.

When Max first got his computer for his 7th birthday I installed it with Debian and since Sarge hadn't been released then it ended up running Sid, and occasionally getting updated. Since it was 'unstable', of course from time to time things would stop working and he would complain and I would fix it and so on.

That computer then went to his younger brother this year when I got him a slightly better one. With Max we were all careful parents, rationally deciding that he was not going to get one until he could read and understand what he saw on it. All very nice. With Fraser the decision came when we could no longer stand all of the arguments and screaming that happened when he wanted to use Max's computer. The reasoning is always different for the second child, and I've heard it evolves even further for subsequent ones.

Anyway, Fraser got Max's old computer running Debian unstable, and Max got a new one. Since I'm generally using Ubuntu for desktop PCs where I don't want to get too bogged down maintaining them I installed Dapper on it, but this has proved problematic. I still occasionally update Fraser's computer, and it sits in the living room so inevitably ends up being used by both of the boys and Max has been getting green recently with all of the new software appearing on Fraser's machine while his has been completely static. He also complains that he likes to see all the text scrolling past when it boots - not that fancy boot loader screen that Ubuntu comes with, so in the end he persuaded me to upgrade it from Ubuntu to Debian.

The upgrade from Ubuntu to Debian is something I have done before, and I knew it was a non-trivial exercise. I think I got it roughly right this time, and I'll add any further gotchas to this story as I discover them from time to time in the coming days.

The first thing to note is that it can take a long time. Although it took only 20 minutes to download the 1000 packages that were going to be reinstalled (since the nice man came and fixed my internet connection yesterday :-) the actual installing of them was slow.

Significant untangling of dependencies was needed. I firstly did something like:

dpkg --purge `dpkg --get-selections | grep ubuntu | cut -f1`

which got rid of all of the Ubuntu meta packages and sounds and themes and so forth which were going to make life impossible if I left them around for too long.

I then downloaded all of my replacement packages, attempting to just 'dist-upgrade' to Sid. This was probably a mistake. I think a better approach would have been to purge masses of packages, down to some bare minimum, upgrade that bare minimum and then build it back up - possibly using Fraser's computer as a template package list.

Oh well. This is about what I did, not what I didn't :-)

So I found myself in a loop of:

apt-get -fuy dist-upgrade
apt-get -f install
dpkg --configure --pending
dpkg --purge this-package
apt-get install this-package
dpkg --purge `deborphan`

for some time. Quite a lot of time, in fact. I would gauge my progress by periodically counting the packages with 'ubuntu' in their version number:

COLUMNS=200 dpkg -l -O | grep ubuntu | cut -c5-94

and eventually my dist-upgrade appeared to have succeeded, although there were still over 100 packages with 'ubuntu' in the version number. Past experience told me that this meant the job was not complete, and indeed the other computers around here running unstable all had around 0-1 packages like that.

The next trick was to:

apt-get --reinstall install `dpkg --get-selections | cut -f1`

which doesn't work, in fact, because some things have circular references and so on. A better solution was to:

dpkg --get-selections | cut -f1 >to_upgrade.lst
vi to_upgrade.lst
(delete all the ones that can't be handled)
apt-get --reinstall install `cat to_upgrade.lst`

and that worked reasonably well, but I think that last line would have been better as a loop:

for PKG in `cat to_upgrade.lst` ; do
  apt-get --reinstall install $PKG
done

One thing was still mystifying me, and that was "why is 'lsb_release -a' still telling me this is Dapper"? Well, it looks like Ubuntu's lsb_release sticks a file in /etc/lsb_release containing this information, and that Debian's lsb_release honours the presence of that file, but doesn't manage it's existence. Once I deleted it that looked better.

Believing everything was working now, it was time for a reboot. This proved to be a bad decision. During the install I had been upgraded to a 2.6.18 kernel and something got lost in the translation so that modules were not being loaded correctly. Or something.

So I built a new kernel on another box, this time making sure that the ide drivers were built-in to the kernel and it would be able to mount it's drives. I then booted from a live CD, installed the kernel and tried again.

It all worked now, except that I needed to compile some binary drivers for the video card, and I'd built that kernel on Heather's system (because it's a dual core AMD64 :-), which is running Dapper and now I needed to build it again on Max's computer so we could build the modules there against the right packages and so on. On a PIII 900 it just takes a bit longer.

A little more mucking around and everything seems to be kind of working, or getting there. I've warned Max that it might be a day or so before everything is working correctly, as he expects it to be. I know sound isn't fully correctly working quite yet but he said that's not important as he doesn't use the sound anyway, which was interesting.

However his 'killer app' is now caught up with the version on his brother's computer and all is right in the world.

He spent the next few hours checking out all the cool new features in the latest TuxPaint, while I got back to what I would rather have been working on all along.

Just before I went to the Sydney Moodle Conference I set up a sourceforge project for the Really Simple CalDAV Store. This is good because it disassociates the project from Catalyst to some degree as well as providing me with space for forums, bug tracker and so forth.

It's interesting to look at Sourceforge from this viewpoint though, because I wonder how lively sourceforge really is. According to their home page they have "Registered Projects: 132,439 Registered Users: 1,421,324", but after a few days activity with my RSCDS project I have managed to surpass 132,319 of those projects to be at number 120!

Is it really that easy to be more active than those 132,000 projects? It would seem so. Penny, Martyn and Nigel have also set up a project last week for the Mahara ePortfolio which they are working on, and at this point there is almost nothing there at all. Even so, that project is ranked at number 18,000 or so, suggesting that there are around 110,000 projects on there which are basically dead.

Still, it seems that it might be something to do with their weighting algorithm. I took a look at the statistics for Project #121 (Sahi at the moment) and it seems to have a noticeably greater number of web hits, downloads, and so forth, so I can only conclude that my secret weapon has been my mirroring of my Git repository into the Sourceforge CVS and Subversion repositories.

Well, to be strictly correct, I guess only the mirroring to the Subversion repository counts, since there are no statistics available for the CVS repositories it seems unlikely that they enter the mix in this way. So with that in mind I feel I should level the playing field and publish my mirroring scripts.

So here they are, for mirroring from Git into a CVS repository use cvs-mirror-git-init first and then use cvs-mirror-git whenever you want the mirror to be brought up to date. Hopefully the 'usage' information is sufficient to describe the care and feeding of the CVS mirroring scripts.

For mirroring from Git into a Subversion repository use svn-mirror-git to bring the repository up to date. Before using it you need to do something like:

mkdir newproject
svn add newproject
cd newproject
svn propset svn:ignore .git .
svn commit

to create yourself a new empty directory for the project to exist in.

I'm currently at the Sydney Moodle Conference, which has (of course) been a great event. Everyone who knows me knows how much I enjoy going to these open source conferences such as the various Moodle "Moots", Debconf and the Linux.conf.au annual Australian Linux conference. I think that the people at these conferences must crave these occasional face to face meetings with all of the people they interact with on-line.

All credit to Julian Ridden and the Monte St Angelo College here in North Sydney for making it so enjoyable. The college is a very nice and photogenic old school within the commercial part of North Sydney and one of the highlights for me has been today's keynote by Diane Brook (who is the ICT head of department at Monte St Angelo) about Teaching the Net Generation. Hearing her talk of the challenges of keeping up with today's school children really brought back to me my personal experience of finding something missing from Wikipedia, adding it, and seeing it turn into a whole fleshed out article over the next few months.

Last night's dinner (sponsored by Catalyst IT of course :-) was a lot of fun, but it's amazing how no matter how many wonderful people I manage to talk to there are still many more who I have not yet met, and I probably won't get to, since the conference ends in a couple more hours, and I'm here writing this blog about it.

So I think I'll leave it at that, and run off and find some more people to chat with!

Congratulations to the Mozilla Calendar project for getting to release 0.3. This is a significant release, and it marks the project's achieving sufficient quality to make it to be mentioned on the front page at Mozilla.

At this point both Sunbird (a standalone application) and Lightning (a plugin for Mozilla Thunderbird) are both quite usable - I have used both for days at a time without crashes. They are, of course, not without their issues but my experience has not involved data loss, or really any wierd disfunction at all.

One of the great features of Lightning / Sunbird, and the reason I have been playing around with it, is that it supports CalDAV calendars, and I have been making my Really Simple CalDAV Store to work with it, as well as adding the facility to maintain timesheets that way in our Work Request Management System.

There are a number of open standards which allow for the potential for storing calendaring information in remote repositories, but the only one that really matters is CalDAV. It is actually a fairly simple standard, based heavily on pre-existing standards like RFC2068 (HTTP/1.1), RFC2445 (iCalendar), RFC2518 (WebDAV), RFC3744 (WebDAV Access Control) which alas, are not quite so simple.

Common existing ways to share calendars generally work around one of (a) just sharing free/busy information, (b) publishing your calendar to a URL (c) sharing appointments via e-mail or (d) doing the whole thing on teh intarwebs. Unfortunately none of these achieve the level of functionality of commercial group calendaring.

CalDAV does manage to provide a framework where those levels of functionality can be achieved because it involves sharing individual calendar entries, rather than whole calendars. It also allows for multi-user access to the same resources and uses standards which generally already inter-operate just fine (such as iCalendar, which is even supported by the Microsoft calendaring applications). In fact most of the CalDAV specification seems really to provide sanity around how to tie all these things together, along with a REPORT method to allow querying of calendar resources with some knowledge of the kinds of data they actually are.

CalDAV will also get another notable boost in a month or two when Apple finally release OS 10.5, which also includes a calendar application that supports CalDAV to partner with their free open-source calendar which is already available.

So I heartily recommend downloading either Sunbird or Lightning and seeing if they will do the job for you, and if they do you may be able to use an on-line server to store your calendar data as well. Of course if you do use it with an external calendar server there may often be less concern if your calendar client does crash, since the data will still be on the server.

Thunderbird and Lightning not very, very frightening at all!

My Really Simple CalDAV Store moves on apace. I have rewritten several of the libraries to use a more consistent structure, and have now implemented a fair subset of the PROPFIND command - to the extent that Mulberry now works with it as well as Sunbird/Lightning and Evolution.

I've also started developing a regression testing framework, and I believe that the database structures around the calendar resource data are now basically correct. From this point I will provide patch scripts to allow upgrade of resources.

It is interesting to see the different approaches that client software has taken to dealing with CalDAV, and I have to think that this has something to do with the long process of evolving the specification. It will be nice when it is complete, and people can develop against something that isn't moving. I believe that the CalDAV specification is actually quite good and straightforward, but I don't feel nearly as happy about the iCalendar spec.

The iCalendar definitions of timezones, on the other hand, have to be one of the best examples of overachievement out there! It is not surprising that none of the software I have come across actually includes more than the most recent timezone definition information within events. The full definition of the New Zealand timezone comes to around 15k, and it would be ridiculous to include that with every event! In an even more bizarre twist, there is no field within the VTIMEZONE which references the timezone with a standard name! No doubt that is not problematic when most people are arranging things within a timezone, but for global events it really is a significant flaw. Evolution uses an X-LIC-LOCATION property to reference the Olson timezone name, which seems to be an industry standard of sorts, but the standard should define such a thing. Even better, having defined a standard way of referencing the timezone name there would be no need to schlepp the full timezone redundantly along with every event.

Well, enough ranting :-) Improvements in this version of RSCDS include:

• Fix some bugs in caldav-REPORT, which was not working with Lightning.
• Complete work on PROPFIND so that Mulberry now works.
• Add MKCOL, which is based on MKCALENDAR, to support hierarchies of collections better.
• Rewrite REPORT to use the new XML libraries.
• Commence support of relationships and permissions.
• Write new ics.php which allows export of the full repository (for an admin), or a subset of the repository.
• That new ics.php allows webcal presentation of the calendars also, so that even if evolution can't support tasks as CalDAV, it can at least refer to tasks someone else puts there with (e.g.) Sunbird.
• Started development of a regression testing framework.

I think I'm now at a reasonable stage to have a Debian package repository for this project. Anyone interested can browse the Git RSCDS repository or Andrew's Web Libraries repository to see my progress in more detail, it's still better to provide a place where it can all be downloaded from, so you can now add this to your sources.list:

deb http://debian.mcmillan.net.nz/debian unstable awm

For those unenlightened folks running on systems that aren't based around the Debian packaging system I've provided some .tar.gz files for RSCDS and AWL that you can download as well.