Monday, February 14, 2011

The last time we checked in we looked at messaging from a monadic perspective. This time we want to look at a broader class of communication from the point of view of the monadic design pattern. At the heart of this idea is recognizing that what is currently the rage of key-value databases is actually a situation that has a wide range of potential interpretations all captured by a single parametrically polymorphic pattern. Consumers consume data from keyed locations in the store while producers deposit data to keyed locations in the store. The basic control abstraction that supports this situation is the delimited continuation. If a consumer is asking for data from keyed locations where no data exists the store grabs the consumer's thread's continuation up to the point of the request and stores it at the keyed the location. When a producer provides data to some or all of these locations the store first checks to see if there are any continuations waiting on these keys and -- if there are -- makes the data available to them first. This essentially resumes the consumer's thread's flow of control at the point in it's computation where it requested the data from the keyed location.

This basic flow of control abstraction is subject to variation in numerous ways including

varying whether the request consumes the data, removing it from the location or simply passes a copy to a requester

varying whether a stored continuation is removed from the location when data is supplied or simply passed the data

The last variation covers a simple pub/sub mechanism. When continuations are persistent (i.e. not removed on matching data production), this is a simple subscription mechanism. When data is persisted this is storage. When it is not, this is message-passing. When key-matching has a certain level of sophistication this arrangement supports basic broadcast capability. If you look at the code sample and the traces in the screenshots you will see that we've used regular expressions as a basic matching. In the specialK repo we use unification.

All of these variations can be captured in a single configurable mechanism -- the configuration of which can be treated as policy governing communication between producers and consumers. To my mind this is really the pattern underlying the web. The web, in microcosm, is a parametrically polymorphic key-value db. Let's look at some code, however, before getting too philosophical.

if(meets.isEmpty){valplace=representative(ptn)tweet("did not find a resource, storing a continuation: "+rk)registered(place)=registered.get(place).getOrElse(Nil)++List(rk)rk(None)}else{for(placeNRrscNSubst<-itergen[PlaceInstance](meets)){valPlaceInstance(place,Left(rsrc),s)=placeNRrscNSubst

waitlistmatch{// Yes!casewaiter::waiters=>{tweet("found waiters waiting for a value at "+ptn)valitr=waitlist.toList.iteratorwhile(itr.hasNext){k(itr.next)}}// No...caseNil=>{// Store the rsrc at a representative of the ptntweet("no waiters waiting for a value at "+ptn)channels(representative(ptn))=rsrc}}}}

defmput(channels:Map[Place,Resource],registered:Map[Place,List[RK]],consume:Boolean)(ptn:Pattern,rsrc:Resource):Unit@suspendable={for(placeNRKsNSubst<-putPlaces(channels,registered,ptn,rsrc)){valPlaceInstance(wtr,Right(rks),s)=placeNRKsNSubsttweet("waiters waiting for a value at "+wtr+" : "+rks)rksmatch{caserk::rrks=>{if(consume){for(sk<-rks){spawn{sk(s(rsrc))}}}else{registered(wtr)=rrksrk(s(rsrc))}}caseNil=>{channels(wtr)=rsrc}}}