Oracle Blog

My thoughts on design and technology

Pubsub sender-side Email storage concept

After posting my thoughts on Twitter and Email and reading comment on it, I've been thinking a lot about how outdated the store-and-forward mechanism is for modern asynchronous communication, and how Email could be advanced to meet current and future needs.

From the very beginning, Email delivery has been modeled after postal mail: a sender creates some content and ships it away, where it gets relayed with every step closer to the destination, until it arrives at the designated mailbox. This technique is referred to as "store-and-forward," and SMTP was designed this way 25 years ago because the network back then was unreliable and nodes were not online all the time.

Today, these are no longer the cases. Recent trends such as blogging, podcasting and twittering take advantage of these changes and make use of a completely opposite paradigm—publish-and-subscribe, also known as pub/sub, in which writers publish content on a server, and readers subscribe to those content via feeds. Similar to radio, there are all kinds of broadcast signals in the air but as listeners you pick which station to tune into.

To a certain degree, an Email sender is the same as a blogger in that both publishes content. The difference is a blogger doesn't send what she wrote to others because they, as subscribes of her blog feed, will come to her website to pick it up. Could the same concept be applied to Email whereby a sender publishes an Email on a website which creates a permalink and updates the feed, then the named recipient pulls down the Email through subscription?

Although I came up with the idea independently, I wasn't the first. Upon research, I found out that a sender-based storage concept coined IM2000 was originally proposed by professor Daniel J. Bernstein (djb) about 10 years ago. Where IM2000 differs from mine is that it's not truly pub/sub because tiny notifications are sent to the recipient.

There appears to be many merits to a pub/sub sender-based storage Email concept. For example, it puts the burden of storage on the sender, not the recipient who may have a disk quota from her service provider. Should someone send you a 20MB attachment and you're already at 95% of your 100MB quota, you don't have to worry about the message bouncing because there's not enough space for it. But if you're the sender and your quota's at 95%, it's your responsibility to upgrade your account to one with bigger quota, and in the meantime you can still receive messages.

Pay for what you do, not what others do to you. Seems fair to me.

[UPDATE Nov 21, 2007] Note to self: there is a few issues with sender-based storage:

@zdz thanks for the pointer. It appears NGMP (that's the experiment you referred to, right?) started in 2003 and ended in 2005. Using XMPP as transport is an interesting choice because I think of XMPP as a synchronous communication protocol and Email is asynchronous. Will look into it some more, but if anyone knows how happened to NGMP, please leave a comment.