Data Portability and the File System

With an increasing dependence on distributed software, and web-based applications the portability of personal and corporate data is becoming an increasingly important issue for all users, but more so for web workers in particular.

Open Data philosophies have begun to coalesce around essays such as the speculative Data Bill Of Rights and the emerging Data Portability movement, web-based services that support portability are still quite rare and invariably the exception to the rule.

Advertisement

Services such as Flickr, del.icio.us and Gmail do allow data extraction of sorts; indeed Gmail’s support for IMAP was apparently motivated by the desire for data portability and enabling users freely import and export messages. Conversely, Microsoft announced that it would end offline Outlook support for Hotmail, effectively imprisoning user’s messages inside Microsoft services, without even a paid for option for IMAP or POP access.

Technicalities aside – portability is really about ethics and ownership. In an marketplace where users are directly contributing assets to the success of a service, we need to be able to assert ownership over those contributions and demand mechanisms to support that ownership.

The component technologies and infrastructure exist and are in place to make this happen – iSync, AIR, Creative Commons, APML, OpenID, etc. – but the demand from users and pressure on service providers isn’t strong enough yet.

Web workers have been the early adopters of this culture and I feel we should be at the forefront of demanding change amongst service providers. There’s a leadership for this community to take in improving web culture for everyone.

The Data Portability group seeks to create ‘a distributed file system for data, bringing existing open standards together’ but the user experiences remain undefined, overtaken by the necessary development of standards and practices. However, perhaps ‘file system’ is the key to understanding what this user experience might be…

This week saw the release of Flickfs (originally reported by LifeHacker), an application that lets Linux users mount their Flickr account as a virtual filesystem, enabling users to drag & drop photos between their desktop and their Flickr account. Notably, the metadata associated with each photo is rendered as an attached file.

Orienting data from applications around the desktop is actually a very smart move – enabling application developers to focus on simply providing mappings between virtual files and data, letting users use tried and trusted desktop metaphors and mechanisms to backup, duplicate, copy and migrate their data.

I’m trying to kick off a Python project — SocialFS — that will attempt to address a lot of these issues by providing a generic data store and the facilities to hook into web app APIs or be used as a normal file system (with some pretty kick ass properties). I’ve started writing out my thoughts about how it all would fit together here…

If you’re into the whole Open Web idea, I’d love some guidance. And if you have some Python and/or FUSE chops, I’d love some help…