Finally, I have released DAViCal 0.9.5. Hopefully this will resolve the series of installation- and upgrade- related problems which plagued the 0.9.4 release.

Thanks for everyone being patient while this release was thoroughly tested through five pre-release versions, and especially thanks to those patient people who helped test those pre-releases.

Now if I don't get too distracted by:

• processing my GPS tracks of the last two weeks around the South Island
• Uploading my last six months worth of photos
• Making sure the kids have done their homework
• Describing the places near where I live.
• ...anything else that catches my interest

... then maybe I will be able to really concentrate on nailing the scheduling extensions work over the next couple of months...

Wish me luck!

Quite a few people seem impressed with the new release of Leopard, and are now looking for a CalDAV server to use with their shiny new iCal app. Unfortunately it seems that Apple wrote this primarily to work with their own (free, open-source) calendar server, which has the side effect that it doesn't work with DAViCal.

Part of the "doesn't work" is due to DAViCal not implementing some areas of the CalDAV specification, which is fair enough. Part of it is due to DAViCal not implementing some draft extensions to the CalDAV specification, which I can also understand, since it allows them to provide some useful features that those extensions are designed to support. There also seem to be some parts of the "doesn't work" which are due to a dependence on extensions beyond either of these cases, which is a little more disappointing - and quite a bit harder to implement.

So far I have made some fixes to the first point, and some additions towards parts of the second, but as of today it still does not work. This is complicated by my not having access to a Mac. Things are looking up, however, because Tom Robinson has kindly agreed to loan me a Mac running Leopard from next week.

In order to "clear the slate" for that, I will be releasing a 0.9.2 over the weekend with the various minor enhancements and fixes that have been applied over the last week. So although this upcoming release will let you add your DAViCal account to iCal 3, it still won't actually work with it. I'm hoping that ready access to the application will enable me to correct that fairly quickly.

Also waiting in the wings (and which unfortunately won't be in 0.9.2 either) Maxime Delorme has been working on SyncML support, and is nearly ready with a patch, so we can look forward to that addition fairly soon also.

I have not been able to put a lot of effort into DAViCal over the last couple of months, since my father was diagnosed with stomach cancer in early September, and he died on 2nd October.

So here it is, finally, including a lot of refactoring work around the handling of DAV/CalDAV REPORT requests and implementation of the DAV::principal-property-search report. This also requires an upgrade to the latest AWL library (0.20), which includes a complete rewrite to the class used for parsing and rendering iCalendar data.

This release is recommended, since you will need some of this stuff to support the upcoming Mozilla Calendar 0.7 release properly.

At this point I have only released the files to http://debian.mcmillan.net.nz/ and I'll push it out to a wider audience if I don't here screams of anguish from people in the next few days :-)

This release does not have any associated database changes, so it should be a simple matter to install the upgraded code.

After much wading through possible names, none of which really excited me, I have finally chosen "DAViCal" as the new name for my CalDAV server that was previously called RSCDS, or the "Really Simple CalDAV Store".

In the end, I chose DAViCal because it:

• seems easy to pronounce.
• combines the 'Cal' and 'DAV'.
• returns < 1000 results on google.
• doesn't make me cringe.
• didn't have a domain name registered.

That was about the hardest part of preparing for the 0.8.1 release, and now that I've done that I should manage to make the changes to the packaging, though I have no doubt that the old name will appear in all sorts of places for a while yet.

Choosing names is an important business, and I should know that from the length of time we spent agonising over names for our children, discarding all sorts of things because they had silly abbreviation collisions (like the "Royal Scottish Country Dance Society" :-) Even then, I think we got the kids names wrong, and the big one should be called "Thumper" with the little one called "Sly", but perhaps that's just a temporary annoyance and in time the names that we registered for them will fit them better.

I also recall Grant once saying that you should never use the word "Simple" in the name of your project, and he should know. DAViCal is no longer particularly simple, although I have attempted to hide the complexity from the user as far as that is possible, and will continue to do so.

Once I get out version 0.8.1 of DAViCal I will finally upload it to Debian, proper. This version has some important enhancements to its DAV spec compliance which are going to be needed by some future versions of Mozilla, and probably other things too, so it's important to push it out as soon as possible now.

Having taken time out to go to the other side of the world to attend DebConf 7 in Edinburgh, and then after spending some time with my family, I have been somewhat distracted from things CalDAV for a wee while now. There are signs that I will be able to put a few more evenings and weekends into it in coming months.

Current Status

With the release of 0.8.0 I believe it is quite a usable application for shared group calendars. We use it at Catalyst for about 80 staff and, while we're not heavy calendar users, it manages without any noticeable performance problems.

There have been an impressive number of downloads from Sourceforge, as well as substantial downloads from my Debian repository, and I believe that the program is in tens, if not hundreds, of installations.

Future Plans

In the immediate future I want to get the basic framework for querying properties about the owner of a calendar or a set of calendars. This will be used by Mozilla soon during their configuration of a new CalDAV source, and I think that Apple iCal probably uses them already. This is partly done, but it's implementation has been slowed by the necessity to refactor the handling of the REPORT method in general to make the code more modular and readable, and to support queries and properties in a more general manner.

The refactoring of the REPORT method handling is in place now, so moving on to the handling of DAV:principal objects should become fairly straightforward. I may end up with a fair bit of quiet, undistracted time over the coming weekend so if I can get in the 'zone', it may get done, and we can start rolling towards a release 0.9.

Release 0.9 will have a new name. One from the cast of thousands that have been submitted to me so far, or one that I will hastily think up in between, but the next released version will not be called RSCDS any longer.

The main goals for further into the future are:

• Maintain interoperability with clients
• Improve specification compliance
• Extend coverage to related specifications
• Documentation improvements
• Configuration improvements
• Installation improvements

Interoperability

I think that RSCDS does fairly well on the interoperability front. Unfortunately I have no access to the preview of OS 10.5, with it's native CalDAV client (though I have been told that it works with RSCDS). Likewise, other commercial CalDAV clients aren't available to me for testing either, so I have concentrated my efforts on supporting the featureset used by the various free CalDAV clients.

My initial work focused around supporting Evolution, in particular, but with the recent effort put in by the Mozilla Calendar team the primary interoperability challenge is to enhance RSCDS to support their areas of development. If I could see how the OS 10.5 client behaves, too, I believe that would help identify a few points to extend, as well as suggesting some future directions.

The best place to review interoperability would probably be at the CalConnect Roundtable 10 but I think that without some sponsorship for airfares I probably won't be able to make that. Maybe next time.

Specifications Compliance

As I read, and reread, the CalDAV (and DAV, and HTTP, and ...) RFCs the compliance with the various specifications improves. Recently I have put a lot of work into the calendar-query report, in particular, which means it now supports querying against arbitrary properties much more comprehensively. This is important because some of the client software is starting to do arbitrary queries also, and it is inevitable that queries for the classification of events, or the percentage completion of todo items will become more common.

And now I notice that there is a new version of DAV in RFC4918, though not too many changes as far as I can see. The intent of this release is mostly to clarify, and slightly to extend, primarily in order to improve client interoperability.

Coverage of Related Specifications

In terms of general calendaring activities, the biggest hole in the CalDAV specification at present is standardisation around the location of other people's schedules. The current draft of the Scheduling Extensions to CalDAV appears to address that, and a lot more besides. I expect to start working on parts of this at some point, but perhaps not sooner than I see a client which starts using it, or when it is ratified, whichever occurs soonest.

Documentation

The documentation is OK. The wiki, in particular, is a reasonable resource for resolving problems (I believe), but it would be good to have some documentation of other things.

One area that could do with enhancement is some documentation of how to check out a copy of the code, and how to provide patches back to me. Even better if other people can maintain their own trees which I could pull patches from, since we're using a distributed version control system (Git) to start with.

Configuration

The administration interface for RSCDS is OK, but could do with a few improvements to make it more usable. Really there are not a lot of reasons to be in the administrative interface, so these improvements should be relatively few, and relatively easy.

This would be a good area for someone interested in helping out to become involved, because it does not entail reading large reams of RFCs, or watching tcpdumps scroll past for the debugging.

Installation

One thing that I have done right, I believe, was to provide the software from the beginning in a packaged form to make the installation easier. Although the software is packaged for Debian, however, it has not yet been uploaded to the archive because I have been waiting for the webapps-common package to enter Debian so I can use that for configuration.

I think I have waited long enough though, so I will upload the next version to Debian for use by a wider audience.

Security

Security has always been a focus for me and I have had the code reviewed for any obvious security flaws. If any particular vulnerabilities are found they will be addressed with appropriate haste.

Retrospective

I've been working on this for a little over a year now, and I think I have largely met my goals for the project. It has certainly been an educational experience for me, and the effect of this work on other projects that I also participate in has also been valuable.

Any new open-source project is always at risk of the developer losing energy and interest, but I believe this one has made it to a state where the bulk of the infrastructure effort has been done, and there are enough interesting challenges in the future to continue to keep me (at least) working on it into the future.

The scope of this project is good, in that it is not at risk of expanding to take over the world. The intention behind this is simply to provide a back-end service for client software which should be very much Someone Else's Problem. In reality there have been times when I have had to get to grips with some of that client software in order to resolve problems, but those have by and large been good experiences.

I see that Jari Urpalainen has written an CalDAV Apache module to support some basic CalDAV functionality.

Excellent.

Meanwhile my own CalDAV server is going through a period of stabilisation before I start in on some of the handling of "Principals", probably next month.

And CalDAV has become RFC 4791 now too.

Here at LCA it seems that some people want shared calendars, so I'm pleaased to be able to tell people that I have spent the last 9 months thinking about and writing a CalDAV server.

While it isn't finished yet (is software ever finished?), it does work just fine with all of the free CalDAV clients that are available, and a number of nice people have translated it into six languages so far.

So if you are at LCA and want to know more catch me at tonight's penguin dinner.

At Catalyst, when there were only 15 or so of us in the office, we used to have a running joke about clients saying "I would have thought", where we would use that phrase to introduce progressively more ludicrous requirements, because we had the occasional client who used that phrase to get work done which had not been written into the contract.

"I would have thought that a system for finding computers on an international network would give some thought to that majority of the planet who don't use the english alphabet.¹

Yeah, sure. That'd be simple, right?

Of course such a phrase really indicates that whether they did (or did not) actually think of that requirement at the time, they certainly never documented it. Either they did think it, and simply assumed that everyone knew their business like they did (and we've all been guilty of making that assumption). Or they didn't.

The joke got to a point where we occasionally found it difficult to contain our giggles whenever a client actually came out with it, and for me at least that continues to this day.

As 'in' jokes go, it was an extremely valuable one to have around though, because it made us notice those points where clients were wandering away from the specification, giving us a decision point where we could clearly either acknowledge our failure to identify the requirement, or to argue whether such an assumption should have been reasonable for us to have understood, and so forth.

I'm reminded of this today, when I read RFC2445, in particular it's definition of PRIVATE vs CONFIDENTIAL.

The document does not give a lot of clues, and seems to trust in the meaning of those words to define which is stronger.

My initial inclination was to go with the ordering of the terms: PUBLIC => PRIVATE => CONFIDENTIAL. That, I thought, seems reasonable. In all the movies the nearly secret stuff is all stamped "CONFIDENTIAL", and then stuff gets stamped "TOP SECRET", and in such movies I'm sure that "PRIVATE" is reserved for signs on doors, rather than on secret files.

I immediately had to reconsider, however, when someone else from the other side of the world (literally) decided that PRIVATE should be the secret stuff, and CONFIDENTIAL was merely "somewhat secret". A small bit of googling suggested that I could well be in the minority on this one, so I will adjust my worldview, correct my software, and possibly even stop watching such silly movies.

Still, I thought it would be nice to deliver these conclusions in the place where they belong: right there on RFC2445. Sure, I could write to the authors and berate them for their wishy-washy language:

"... The access classification of an individual iCalendar component is useful when measured along with the other security components of a calendar system (e.g., calendar user authentication, authorization, access rights, access role, etc.). Hence, the semantics of the individual access classifications cannot be completely defined by this memo alone. Additionally, due to the "blind" nature of most exchange processes using this memo, these access classifications cannot serve as an enforcement statement for a system receiving an iCalendar object. Rather, they provide a method for capturing the intention of the calendar owner for the access to the calendar component."

Oh how very fucking useful. This is supposed to be a specification! The intention of my friend from France when he says "PRIVATE" clearly differs from my own, but that can't easily be arbitrated without a frame of reference which the standard should, in theory, provide.

After I calmed down a bit I was reminded of how some software that I use allows people to annotate the manuals, clarifying things, providing examples and so forth, which I have found incredibly useful from time to time, and it made me wonder whether a site that allowed people to generally annotate RFCs (or other documents) could also be useful.

So: do you think it would it be useful to have some room for interpretation?

---

¹ That one is not a quote, by the way - I have a very good idea of exactly how much debate has gone on around that, and how stupidly complex the issue has become.

I just found some blogging by Lisa Dusseault on CalDAV and discovered that CalDAV is now a 'Proposed Standard'. That may not mean much, but remember that IETF eventually ends up calling these things "Request For Comments". Lisa and others have spent considerable time and effort developing their specification to this point and congratulations are definitely in order.

So: what's interesting about CalDAV?

People have been asking me why calendaring, and in particular CalDAV, is such a hot button for me recently. The answer has evolved over time, and really comes in several parts...

Firstly, my desire for this is not recent

Shared calendaring has been a "black hole" in the Open Source universe for way too long. Several employers ago (in the early '90s) when I worked for ${LARGE_GOVT_DEPT} I used to schedule meetings using ${GROUP_CALENDAR} and it was damned useful. Later I moved on to work for a much smaller organisation, but ${GROUP_CALENDAR} there was still extremely useful - just in a very different profile of use.

When we got together and started Catalyst IT in 1997 I started turning my back on the 'Windows' world, looking at Linux as something that was interesting and Open Source as a movement I wanted to be involved in.

In 1998 I switched full-time to using Linux on my laptop, but in doing so I denied large chunks of application space which simply wasn't available. Heck, my desperate search for a decent open source word processor caused me to be cited in a 'why the concept of XML documents should not be patentable' recently since I really used some early versions of AbiWord to write letters to clients, and I still had them on my hard disk!

Since then I have seen the application space on Linux slowly populate with great programs that do large chunks of what I want to be able to do with my computer. I've contributed to that too, in programming small applications, filing bugs, promotionally and in other small ways.

So Why CalDAV?

After a while people started to talk about DAV, and perhaps this would be a way of providing a group calendar application. Unfortunately the way it ended up being used by most of the applications that did support it was to use it as a place to dump a person's entire calendar as a single file, or maybe just their free/busy time.

That ends up being flawed in a whole bunch of ways: how do you delegate responsibilities among people, for example?

Although DAV evolved over time to answer some of those questions, the applications that were paying lip-service to DAV failed to evolve in step.

A Star is Born

In the calendaring firmament, at least! Lisa Dusseault started to produce specifications for calendaring extensions to DAV and then she got a job with OSAF specifically to do that. That standards development process has taken three years, but we're finally near the end of it. Here's the description of CalDAV from their website:

"CalDAV models calendar events as HTTP resources in iCalendar format, and models calendars containing events as WebDAV collections. This allows users to publish and subscribe to calendars, share them collaboratively, synchronize between multiple users and synchronize between multiple devices.

Which is an excellent conceptualisation of what is (was) needed.

Meanwhile I was still looking around for a solution to the group calendaring problem, and not just for myself: Clients were also regularly asking what could be used to provide them with a shared calendar solution, and if they were asking me that then they were looking for an open source answer.

Now we have a standard, where are the implementations?

At this point it kind of becomes "chicken and egg". There are no servers, so people can't easily test the clients they are developing, and there are no clients, so people can't easily test their servers. Well a few people are working on these things, of course. On the client side, Mozilla is adding CalDAV support and CalDAV support snuck into Evolution in late 2005. Cyrus Daboo (one of the other authors of CalDAV) implemented it in his 'Mulberry' free-but-not-yet-open-source application, and there are a bunch more which are associated with client-side projects as well.

Looking around though, it seems that there are no simple servers. Some of the servers that are being written seem likely to fall into the trap of working best with their own client application, or they are simply proprietary solutions. Or they have requirements that are (to me) not particularly desirable, such as being written in Java.

I don't have a problem with Java, per se, but when you might want to run 20 applications on one server it always seems to want to get bloated and to eat administrator time...

Hence the Really Simple CalDAV Store

Eventually I got fed up with looking around the open source landscape every few months, searching for group calendar servers and finding the continuing promise of jam tomorrow, but no jam today.

Around May this year I decided to build my own, and around 4th of September I finally had enough answers to start work on it.

Yesterday I made a significant milestone release, with support now for multiple languages in the administrative interface, delegated permissions are working in a basic manner and the CalDAV operations used in Mozilla Calendar and Evolution are fully supported. Most of the CalDAV operations in Mulberry are supported too, although some of the more arcane ones will be my work for the coming month.

Thanks!

Thanks to Cristina, Lorena and Guillaume for their help in translating the administrative interface. I hope to thank a few more people for more translations soon also!

And also, thanks to Lisa, Cyrus and Bernard and everyone else who was involved in making the CalDAV idea a reality. Great work.

Just before I went to the Sydney Moodle Conference I set up a sourceforge project for the Really Simple CalDAV Store. This is good because it disassociates the project from Catalyst to some degree as well as providing me with space for forums, bug tracker and so forth.

It's interesting to look at Sourceforge from this viewpoint though, because I wonder how lively sourceforge really is. According to their home page they have "Registered Projects: 132,439 Registered Users: 1,421,324", but after a few days activity with my RSCDS project I have managed to surpass 132,319 of those projects to be at number 120!

Is it really that easy to be more active than those 132,000 projects? It would seem so. Penny, Martyn and Nigel have also set up a project last week for the Mahara ePortfolio which they are working on, and at this point there is almost nothing there at all. Even so, that project is ranked at number 18,000 or so, suggesting that there are around 110,000 projects on there which are basically dead.

Still, it seems that it might be something to do with their weighting algorithm. I took a look at the statistics for Project #121 (Sahi at the moment) and it seems to have a noticeably greater number of web hits, downloads, and so forth, so I can only conclude that my secret weapon has been my mirroring of my Git repository into the Sourceforge CVS and Subversion repositories.

Well, to be strictly correct, I guess only the mirroring to the Subversion repository counts, since there are no statistics available for the CVS repositories it seems unlikely that they enter the mix in this way. So with that in mind I feel I should level the playing field and publish my mirroring scripts.

So here they are, for mirroring from Git into a CVS repository use cvs-mirror-git-init first and then use cvs-mirror-git whenever you want the mirror to be brought up to date. Hopefully the 'usage' information is sufficient to describe the care and feeding of the CVS mirroring scripts.

For mirroring from Git into a Subversion repository use svn-mirror-git to bring the repository up to date. Before using it you need to do something like:

mkdir newproject
svn add newproject
cd newproject
svn propset svn:ignore .git .
svn commit

to create yourself a new empty directory for the project to exist in.