Dancing out of time: Thoughts on asynchronous communication

Why asynchronous communication scales, and what we can do with that power.

I find it useful to try to think clearly about communication systems. This includes the
ways in which they are synchronous or asynchronous (or a mixture), and the
disruption that occurs when new technology allows us to replace synchronous
systems with asynchronous ones, or to deliver new forms of asynchronous
communication.

This post is the first of a two-part series. Below, I take a look at the core differences between synchronous and asynchronous techniques, and I suggest an alternative to traditional synchronous API-based communication between applications. Part two addresses Tim O’Reilly’s question: “Where is the Web 2.0 address book?“

Synchronous and asynchronous communication

Synchronous communication requires that the participating parties — information producers and consumers — are simultaneously involved. Examples abound: telephone calls, in-person meetings, a parent teaching a child a song, an audience listening to a public lecture or a live concert, buyers and sellers negotiating prices at a market, the broadcasting and simultaneous consumption of live events, Morse code flashed between ships at sea, and the Gotham City bat signal.

Communication can also be asynchronous. In this case, information is
somehow — broadly speaking — published and is only later consumed.

The history of technology has many examples of the tremendous impact that
can result when a form of synchronous communication is replaced with
something asynchronous. The most obvious example is the invention of
writing, which allowed information to be written and consumed later.
There are many familiar examples, such as CDs and DVDs, the VCR (and its
modern derivatives like the TiVo and Boxee), voicemail, etc.

There are many other less obvious examples, too. Post-it Notes are
asynchronous: they let us publish information in physical locations that
we choose carefully so the intended recipient will likely encounter them
later. Blog posts, including this one, are asynchronous. There are also plenty of
non-human examples, such as dogs urinating on lamp posts.

Communication doesn’t always divide cleanly into synchronous or
asynchronous. Many forms of communication have some elements of both. How
would you classify the Pony Express, smoke signals, sky writing, and
telegrams? Susan
Boyle’s performance on “Britain’s Got Talent” was experienced
synchronously by a few thousand people, and asynchronously by 50
million. And when Charlie bit his brother’s finger (again) in the back seat
of the car, the event was experienced synchronously by just three people,
but later, asynchronously, in 233 million online viewings.

The requirements of asynchronous communication

The main requirement for asynchronous communication is a mechanism for
interim information storage. This can take many forms: wet concrete, the
soft bark of a tree, a wall, or one’s own skin. Along with storage, one
must also have a means by which to deposit information. In our previous examples that would be a finger, a pen-knife, spray paint, and a tattoo needle.

It’s difficult to overstate the impact of technology on our ability to
communicate asynchronously. Modern digital systems provide us with
virtually unlimited amounts of cheap storage, along with the means to
efficiently deposit/retrieve information into/from this storage. Consider Slideshare as an example: You can post presentation slides and have them read asynchronously
by thousands or even millions, rather than just seen by the dozens who might attend a presentation in person.

The other requirement is a set of conventions about the communication.
These may be high-level and taken for granted. For example, the language of
communication. Conventions are often more explicit, and can become
specialized over time, like apartment ads: lg 3br apt, d/w & a/c, hdwd
flrs. Conventions can also take the form of protocols, in which parties
exchange information using the shared storage. Examples include quoting conventions in
email threads, and people using @name to address others and check for
mentions in Twitter.

Asynchronous communication lends itself to scale

Asynchronous communication almost always scales better than
synchronous. Synchronization is usually centralized, complicated, and
expensive because participants are often forced to waste time waiting.

The main requirements for asynchronous communication are things that
modern technology is providing in abundance. Experiences that were formerly limited to a small set of synchronous participants are now regularly experienced asynchronously by a significant fraction of the people on the planet.

At smaller scales, there are many familiar examples — including
non-digital ones — of how asynchronous communications scale so easily and
cheaply. Putting up a billboard beside a busy highway is a cheap and easy
way of delivering a message to many. Using “Wanted” posters is better than
having the sheriff stand on the corner describing a fugitive to passers
by. Having a mailbox in front of your house provides temporary mail
storage, which ironically can be faster and more convenient than a
synchronous meeting with a courier service who will put your package back
in their truck if you’re not home to sign for it. Higher tech examples are
abundant, as mentioned above.

Because asynchronous systems scale so much better than synchronous ones,
coming up with ways to replace the latter with the former can result in the
creation of extraordinary value. For example, consider Twitter and YouTube.

APIs, data protocols, and FluidDB

My main interest in thinking about the possibilities of asynchronous
communications is in the area of how applications communicate (and might
come to communicate). This is often done via APIs that applications
provide, and in much of the computational world communication between
applications takes the form of synchronous calls to these API methods. I
believe this presents an opportunity for an asynchronous alternative, and
that the alternative can have an impact similar to those we’ve seen
historically when synchronous systems are replaced by asynchronous ones.

In particular, I’m currently involved in building FluidDB, a new kind of
database under development at Fluidinfo. Among other things, FluidDB aims
to provide shared always-writable storage that applications can use to
exchange information. [Disclosure: Tim O’Reilly is an investor in FluidInfo.] I believe that predefined API calls between applications are generally
less flexible and powerful than asynchronous communications.

In the second part of this series, I’ll give an example of asynchronous flexibility by offering an answer to Tim O’Reilly’s question “Where is the Web 2.0 address book?” His question arises from a frustration with the
lack of communication between applications. Application-independent shared
writable storage and simple conventions for communication look to me like
the best way forward.

Thanks Terry. This goes a long way to describing my frustration with synchronous availability, and strong preference to asynchronous messaging. I suggested a dynamic protocol description language last year as an alternative to hard APIs and protocols. Curious to see how you envision FluidDBs role in empowering asynch communication between services.

Glen Koskela

Good thread. We have all heared much about convergence of IT and C, many years. Yet I have the feeling that most software engineers have not captured the essense. Traditional one interface (hard API) for a single service for many users, over a request-response model using data structures, associations and one-way (protocol) initiatives still preveil. IT is made out of data and algoritms while objects are passive.

Asynchronous communications allows fully different set of “genes” to be used to build new kind of IT driven applications and services. Objects [things] are active with context [tags], and control stages [permissions], (multi)channels [namespaces] replace direct associations, p2p over one-way-only initiations, shared stored interactivity instead of communication by invocation, etc. One interface with many services while objects and users have simultaneous interactions and services.

What we have been missing is a brodly understood and available software engineering approach so that applications could define the role and objectives for communication [activity], work with shared active objects [social information] of any kind.

In your case I guess this means allowing an application to do much more than what it inherently, with its own information and hard APIs alone, could do. Linking objects with tags to other objects with their own tags over a shared database.

This is really good. Thanks for framing things we see everyday in these fundamental concepts. You also bring up a really good point: communications between software systems are often still done synchronously and would be better asynchronous. We’ve definitely seen it with data integration projects and with the need for looser couplings in software systems in general.

I didn’t realize it at the time, but I used to do Asynchronous communication between BAT files back in the day (wow I dating myself). I would leave little TXT files lying around with the results of Batch routines for other routines to pickup and read when they got around to it. Who’d a thought I was cutting edge ;-).