Calendar:Cache

Contents

Calendar Caching

The main goal of the cache is to speed up queries. All the roundtrips needed
for caldav or groupdav can make the usage of calendar be very slow.

Additionally, it should be possible to access and modify calendar data while
offline. If the user can modify data while offline, there should be a log of those changes that can be applied to the remote (authorative) store when coming back online.

Different, but related, problem is the problem of using one single ics file on a (webdav) server simultaniously from multiple clients. When trying to upload the ics file it might turn out that the external file has been modified. In that case, the changes made after the last succesful upload need to be re-applied to the new file, and uploaded again. Again, a changelog. If the cache can take care of this problem too, that would be a nice way to share code.

Filling the cache will depend on the backend-storage. For caldav, the initial
filling is problematic (as in: I don't know how to do that in a smart way yet).
For ics, it's simply done by downloading the whole file and parsing it.

Updating the cache can be done by asking the caldav server for changed items.
At the same time, the changes made by the user that are not saved yet, need to
be send to the server. Conflicts between those two changesets need to be
resolved.

Shared ics uses almost the same scheme. The provider needs to figure out
the difference between the previous file downloaded and the file as it
sits on the server. Those differences form a changeset. This changeset is
used to update the cache and the display. The changes made to the cache ar
applied to the remote file, and uploaded. If the upload fails (maybe because
the user is offline, or the remote file changed in the meantime) the changelogs
are not deleted, but stored to be used in the next try.

This changelog thing sounds a lot like syncing. That's what is really
happening.

In short, the backend needs a way to get the changed items since last upload.
The cache alse needs to keep a changelog. Then those are used to sync the two
calendars.

Interface Changes

The calendar needs to expose a few flags indicating what it can do:

get a changeset

do 'complex' queries

It needs some methods:

Get changelist

Update store and return if it succeeded. (No external changes since last time chagelist was retrieved)

Getting the Changelist

easy for caldav, hard for ics.

Ics can parse the remote file, and while doing that check the last-changed
time of the events with a stored database of (id, lastchange). Using this
change, it can build a changelist and immidiatly remove the non-changed
events from memory.

Implementation

The cached data is stored in a storage calendar. The cache engine is a wrapper around the storage.

Two models are possible:

The storage represents the last known state of the autorative calendar, and changes made by the user are queued

The storage represents what the user sees, and changes are directly propagated.

The second model makes querying much easier, so that is choosen to be really used. In this model the cache needs to keep a list of items as they weere before the user changed them, so as they are in the autorative store. This is needed for conflict detection and resolution.

More technical details

The cache is stored as a storage calendar.
storage calendar needs an option to return a different type and uri. (the uri should not point to the cache file, but to the real file)

The auth calendar (ics) can return a list of changes since the last flush. The calendar can flush the current data while checking if the file didn't change since last getUpdates.

And observer watches for changes to the cache calendar, and builds a changelog.
On a timer, this changelog is checked with the changelog from the auth calendar for conflicts. Then the two changelogs are applied to the 'other' calendar.

This observer also fills the cache calendar on initial setup. In the future a better way for caldav can be implemented, where the storage calendar has added knowledge about the auth calendar, and can query it.

Notes from Toronto meeting discussion

caching calendar/offline calendar/device sync

history:
mvl/dmose came up with basic design for using storage calendar to hold
offline calendars

already implemented by wcap. could be genericized so it could be used
for all calendars