The iCalendar chicken-and-egg conundrum

Publishing calendars as HTML is necessary but not sufficient. We also need iCalendar feeds.

If you’re running a website for a school, or a business, or a band, or a club, there’s probably a tab on your site’s home page labeled Events. The calendar that shows up on that page is most likely driven by some kind of content management system that collects your events using a form, stores them in a database, and renders them through an HTML template to produce your events page.

One of the premises of this series is that publishing calendars as HTML, for people to read on your events page, is necessary but not sufficient. You should also want to publish events as data for computers to process and for networks to syndicate. And you should expect your content management system, which already has the data in a machine-friendly format, to do that for you. But this almost never happens, and so we have a classic chicken-and-egg conundrum. Nobody expects that a website’s events page should offer a parallel data feed, so nobody demands that CMS vendors provide one, and as a result hardly any do.

Consider the case of the Hannah Grimes Center in Keene, NH. Its events page is provided by the Constant Contact service as part of a package of event marketing features that include registration and social media promotion. But there’s only an HTML page; no companion iCalendar feed is available. That means the Hannah Grimes Center’s events aren’t available as a stream of data that can syndicate through networks and merge with other calendars.

The irony here is that a previous version of the Hannah Grimes site did provide an iCalendar feed. In that version, the events page was served by Drupal. It has a calendar module that publishes event data two ways: as HTML, and as iCalendar. Last week I happened to meet the site’s coordinator; she wondered why their event stream had dropped out of syndication. I explained, and she wrote back the other day saying: “We’re in the process of converting back to Drupal for events!”

Drupal is a great system, and that’s fine, but it shouldn’t be necessary. Every CMS that produces an events page should also produce an iCalendar feed. When I talk to CMS vendors they confirm what I’ve said here: Customers don’t ask, so vendors don’t provide. What I also hear from them, though, is that producing an iCalendar feed seems like a lot of work. It isn’t. Libraries are available to simplify the translation from programming-language data structures to iCalendar feeds. And even without such libraries, it’s not hard to whip up a simple iCalendar feed.

My vision for a network of iCalendar feeds is inspired by the early blogosphere’s network of RSS feeds. Content management systems, like Radio UserLand, produced those feeds automatically. But many of us also produced our own feeds without the aid of kits. We did it “by hand” — that is, by interpolating variables into XML templates. It was easy to do, and it worked well enough to enable our homegrown feeds to participate in the emerging RSS ecosystem. In this week’s companion article (“How to translate an OData feed into an iCalendar feed“) I show how the elmcity service uses a library to produce iCalendar feeds. But if you look at the sample feed at the end of that article, you’ll see that — like the RSS 0.9 many of us created by hand — it isn’t really rocket science.

There was, of course, another reason why the homegrown approach worked as well as it did. It’s true that we didn’t have tools to help us write RSS feeds, but we did have a tool to validate them. The Feed Validator, originally created by Mark Pilgrim and Sam Ruby, was — and remains — a pillar supporting the RSS (and now, RSS/Atom) ecosystem.

When I set out to bootstrap an iCalendar network, I realized that there was no analogous service to validate iCalendar feeds. So I reached out to Doug Day. He’s the author of DDay.iCal, which is the open source library that the elmcity service uses to parse the iCalendar feeds it gathers into each hub, and also write iCalendar feeds that merge all of the inputs to each hub. Doug thought there should be a validator for iCalendar feeds, so he stepped up to the plate and created one at icalvalid.cloudapp.net.

If you’re a school or a business or a band or a club whose website sports an Events tab that doesn’t offer a companion iCalendar feed, I hope you’ll ask your CMS vendor why not. If you’re one of the vast majority of CMS vendors whose systems create events pages but don’t produce iCalendar feeds, I hope you won’t wait to be asked. With or without tool support, it’s not hard to make them. (See this week’s companion article for a C# example based on Doug Day’s DDay.iCal library.) And, thanks to Doug, there’s now a validation service to help you make them right.

Another option if your CMS can’t explicitly generate ics files is to add in microformats or other semantic technologies right into the HTML template. Then you can link to web services which will convert the HTML into ics for you. It is RESTful so you can create standard links even when your CMS doesn’t support custom output or mimetypes.

Thank you for posting this! A coworker and I built an events calendar system from scratch this summer, and one of the features that I insisted on was iCalendar and RSS feed support (http://www.utexas.edu/law/calendar/feeds/). There was befuddled head scratching when I mentioned iCalendar and .ics to our end users, but given a demo on how to subscribe using Outlook or Google Calendar it became an obvious must-have feature. If users aren’t asking for iCalendar, maybe it’s because they don’t know what it is exactly — possibly a problem with the name (confusion with Apple’s iCal app?) or maybe the idea of subscribing to feeds is still alien to our users (even RSS hasn’t fully taken off around here).

I’m very grateful to Doug Day and the icalvalid app, though. That tool saved me a lot of grief while I was working to make sure the data was coming through in the right format!

The irony is that even without the external benefits, any existing CMS would be _improved_ by a modularization that delivered an ical feed from the server that got rendered using ajax or xslt on the client. It would cut server load (to render pages) and improve the interactivity and responsiveness of the user interface. Not to mention the direct benefits of modularization for future management of the CMS codebase. This is true of lots of the data visualizations delivered today on the web—a point we made in a paper at last year’s WWW conference:http://db.csail.mit.edu/pubs/sync-kit.pdf

iCalendar and its related microformat, hCalendar are great but if we want this data to really be useful to people running a service dealing with more than one city/country we need a standard way to specify the city/state/country without having to rely on reverse geocoding from coordinates (often inaccurate for locations near borders) or needing to attempt geocoding from a free-form text location (even worse).

also – timezones (and the common misuse of) are a nightmare!

I have an events listings site and have had iCal versions of the listings on there for years (as well as hCalendar markup on the listings pages).

I also have an experimental feed aggregator where people can submit iCalendar feeds, pages with hCalendar in them, RSS with hCalendar in html descriptions and feeds in various other formats for inclusion in the listings, but the reason this is still “experimental” and events may take days to appear in listings is because of the problems I mentioned above. I have to still check them and attempt to correct wrong cities and dates and remove data where cities and dates are missing or ambiguous!!!)
For now I tell people to use their local timezone (or floating dates) and CLEARLY specify the city, state AND country in the location so that the geocoder has a good chance of getting it right!
(and I also tell them that anything with the ‘Z’ timezone will be discarded – too often misused)

I would love to let people submit events directly from their favourite calendar application but I need that city/country information!

If you have events on your site and want to make this data available for applications and services then you *should* use a real calendar format like iCalendar or hCalendar.

Regarding “confusion with Apple’s iCal app” … that is one of many applications that can read iCalendar data.

For example, I use “Z” and mean it, although I agree that for geographically-located events one wants to use local times and provide the time-zone offsets. Noticing how many years it took Outlook to now allow me to record flight departure and arrival times in different time zones, and manage international conference-call schedules (something that needs to be given in “Z” so I know when to be on the call from wherever I am).

I am still frustrated about multiple calendars and how to merge them. Some appointments I receive always end up in a new calendar and I have to drag them to my working calendar, for example.

I fear this is going to get worse before it gets better, as I am now looking at actually using a calendar on my and Vicki’s new smartphones and dealing with how we may be dragged deeper into Windows Live and Facebook than we are comfortable with. A lot to figure out here, especially since we are devoted Outlook desktop (not Exchange) users.

Meanwhile, I am inspired by the grassroots effort that seems to have shown up in this look at calendar-information portability in and out of content-management systems. Bravo!

The Joomla CMS relies on 3rd parties to provide iCalendar support. Zap Calendar ( ZapCalendar.com ) is a 3rd party Joomla application that provides calendar synchronization and downloading of individual events in iCalendar format. The Pro version can pull events from other iCal supported external calendars, so you could create one aggregate calendar from multiple calendar feeds. For example, a branch office calendar could be pulled by the main office calendar to create a corporate calendar of all branch office events.

Mimi M.

The questions I’m working on are 1) what do you want your personal calendaring app to do when you click on a feed? Do you assume that this is a one-time pull or that the events on the subscription will be updated on your calendar? Do you assume it’s going to make a whole new calendar on your desktop? What if you wanted one event actually pulled into your personal free-busy space? 2) Do you recognize any standard icon as being calendar information?