Communicating Event-Loop Concurrency and Distribution

On both the browser and the server, JavaScript’s de-facto concurrency model is increasingly “shared nothing” communicating event loops. JavaScript event loops within the browser (both frames and workers) send asynchronous messages to other JavaScript event loops via postMessage. JavaScript event loops in the browser send and receive asynchronous messages with servers using asynch XHR, and shortly, Server-Sent Events and WebSockets. And server-side JavaScript has a rapidly growing role as the counterparty of these protocols, and increasingly uses event loops to service them.

This strawman consists of several major parts, not all of which need be accepted together.

(Q(p).post(), Q(p).get()) Make asynchronous requests of objects that may not be synchronously reachable, such as remote objects.

(Q(p).then()) Ease the burden of local event loop programming, by reifying the ability to register a callback as a first class value.

(Q.async, yield:) for implicit registration of shallow continuations on promises.

Syntactic sugar

The infix “!” operator: An eventual analog of “.“, for making eventual requests look more like immediate requests.

(Q.makeFar() and Q.makeRemote()) A promise extension mechanism, so that promise handlers can turn local promise operations into remote messages.

Transport independence: Using remote object messaging as a symmetric abstraction layer, hiding the annoying differences among the various transports listed above as well as server-to-server TCP and UDP transports.

(Vat()) An event-loop spawning mechanism for spawning new event loops that run concurrently with the event loop which spawned it.

Worker independence: Using VatAPI as an abstraction layer around worker spawning on the browser or process spawning on the server.

(Vat.evalLater(), there()) Using JavaScript itself as mobile code, so event loops can safely inject new behavior into other event loops

Symmetric Mobile Code: Generalizes from the current use of JavaScript as mobile code sent only from server and only to browsers.

Concurrency Model and Promises

Aggregate objects into process-like units called vats. Objects in one vat can only send asynchronous messages to objects in other vats. Promises represent such references to potentially remote objects. Eventual message sends queue eventual-deliveries in the work queue of the vat hosting the target object. A vat’s thread processes each eventual-delivery to completion before proceeding to the next. Each such processing step is a turn. A then expression registers a callback to happen in a separate turn once a promise is resolved, providing the callback with the promise’s resolution. The eventual send and then expressions immediately return a promise for the eventual outcome of the operation they register.

This model is free of conventional race condition or deadlock bugs. While a turn is in progress, it has mutually exclusive access to all state to which it has synchronous access, i.e., all state within its vat, avoiding conventional race condition bugs without any explicit locking. The model presented here provides no locks or blocking constructs of any kind, although it does not forbid a host environment from providing blocking constructs (like alert). Without blocking, conventional deadlock is impossible. Of course, less conventional forms of race condition and deadlock bugs remain.

Vats

Partition the JavaScript reference graph into separate units, corresponding to prior concepts variously called vats, workers, processes, tanks, or grains. We adopt the “vat” terminology here for expository purposes. Vats are only asynchronously coupled to each other, and represent the minimal possible unit of concurrency, transparent distribution, orthogonal persistence, migration, partial failure, resource control, preemptive termination/deallocation, and defense from denial of service. Each vat consists of

a single sequential thread of control,

a single call-return stack,

a single fifo queue holding eventual-deliveries,

an internal object heap,

and incoming and outgoing remote references.

A vat’s thread of control dequeues the next eventual-delivery from the queue and processes it to completion before proceeding to the next. When the queue is empty, the vat is idle.

const vat = Vat(); //makes a new vat, as an object local to the creating vat.// A Vat has an ''evalLater'' method that evaluates a Program in a turn of that vat.// The ''evalLater'' method returns a promise for the evaluation's completion value.const funP = vat.evalLater('' + function fun(x, y){return x + y; }); // see belowconst sumP = funP ! (3, 5); // sumP will eventually resolve to 8, unless...const doneP = vat.terminateAsap(new Error('die')); // that vat is terminated before ''sumP'' is resolved.// If the vat is terminated first, then ''sumP'' resolves to a //rejected// problem, with// (Error: die) as its alleged reason for rejection.// Once the vat is terminated, ''doneP'' will eventually resolve to ''true''.

The vat object that represents a new vat is local to the creating vat, so that a vat may be terminated without waiting for that vat’s eventual-delivery queue to drain.

The vat abstraction differs from the WebWorker abstraction, even though both are based on communicating event loops, since inter-vat messages are always directed at objects within a vat, not a vat as a whole. We intend that WebWorkers can be implemented in terms of vats and vice versa. However, when vats are built on WebWorkers, in the absence of some kind of weak reference and gc notification mechanism, it is probably impossible to arrange for the collection of distributed garbage. Even with them, much more is needed to enable collection of distributed cyclic garbage. On the other hand, when vats are provided more primitively, multiple vats within an address space can be jointly within the purview of a single concurrent garbage collector, enabling full gc among these co-resident vats. However, truly distributed vats would still be faced with these same distributed garbage collection worries.

The “ '' + function... ” trick above depends on function_to_string to actually pass a string which is the program source for the function, while nevertheless having the function itself appear in the spawning program as code rather than as a literal string. This helps IDEs, refactoring tools, etc. A vat’s evalLater method evaluates that string as a program in a safe scope – a scope containing only the standard global variables such as Object, Array, etc. Except for these, the source passed in should be closed – should not contain free references to any other variables. If the function is closed but for these standard globals, and these standard globals are not shadowed or replaced in the spawning context, then an IDE’s scope analysis of the code remains accurate.

Promises and Promise States

We introduce a new opaque type of object, the Promise to represent potentially remote references. A normal JavaScript direct reference may only designate an object within the same vat. Only promises may designate objects in other vats. A promise may be in one of several states:

(TODO: Revise diagram to replace “unresolved” with “pending” and “broken” with “rejected”.)

pending – when it is not yet determined what object the promise designates,

pending-local – when the right to determine what the promise designates resides in the same vat,

pending-remote – when that right is either in flight between vats or resides in a remote vat,

fulfilled – resolved to successfully designate some object,

near – resolved to a direct reference to a local object,

far – resolved to designate a remote object,

rejected – will never designate an object, for an alleged reason represented by an associated error.

A promise may transition from pending to any state. Additionally a promise can transition from far to rejected. A resolved promise can designate any non-promise value including primitives, null, and undefined. Primitives, null, undefined, and some objects are pass-by-copy. All other objects are pass-by-reference. A promise resolved to designate a pass-by-copy value is always near, i.e., it always designates a local copy of the value.

If a function foo immediately returns either X or a promise which it later fulfills with X, we say that foorevealsX. Unless stated otherwise, we implicitly elide the error conditions from such statements. For the more explicit statement, append: “or it throws, or it does not terminate, or it rejects the returned promise, or it never resolves the returned promise.” Put another way, such a function returns a reference to X, where by reference we mean either X or a promise for X.

Eventual Operations

The existing JavaScript infix . (dot or now) operator enables synchronous interaction with the local object designated by a direct reference. We introduce a corresponding infix ! (bang or eventually) operator for corresponding asynchronous interaction with objects eventually designated by either direct references or promises.

The ... means “and all the normal right hand sides of this production. By “abstract” here I mean the distinction that must be preserved by parsing, i.e., in an ast, but without explaining the precedence and associativity which explains how this is unambiguously parsed. In all cases, the eventual form of an expression queues a eventual-delivery recording the need to perform the corresponding immediate form in the vat hosting the (eventually) designated object. The eventual form immediately evaluates to a promise for the result of eventually performing this eventual-delivery.

“[nlth]” above is short for “[No LineTerminator here]“, in order to unambiguously distinguish infix from prefix bang in the face of automatic semicolon insertion.

Fundamental Static Q Methods

Q(target) -> targetP

Lifts the target argument into a promise designating the same object. If target is already a promise, then that promise is returned. (A promise for promise for T simplifies into a promise for T. Category theorists will be more pleased than Type theorists ;).)

Q.reject(reason) -> rejectedP

Makes and returns a fresh rejected promise recording (a sanitized form of) reason as the alleged reason for rejection. reason should generally be an immutable pass-by-copy Error object.

Q.promise(f(resolve,reject)->()) -> promise

Makes a fresh promise, where the promise is initially pending-local, and the argument function f is called with resolve and reject functions for resolving this promise.

Q.isPromise(target) -> boolean

Is target a promise? If not, then using target as a target in the various promise operations is still equivalent to using Q(target), i.e., the promise operations will automatically lift all values to promises.

Q.makeFar(handler, nextSlotP) -> promise

Makes a resolved far reference, which can only further resolve to rejected.

Returns the currently most resolved form of target1. If target1 is a fulfilled promise, return its resolution. If target1 is a promise that is following promise target2, then return target2. If target1 is a terminal pending or rejected promise, or a non-promise, then return target1.

Additional non-fundamental static Q convenience methods appear below.

Promise methods

Assuming p is a promise

p.get(name) -> valueP

Returns a promise for the result of eventually getting the value of the name property of target.

p.post(opt_name, args) -> resultP

Eventually invoke the named method of target with these args. Returns a promise for what the result will be.

p.send(opt_name, ...args) -> resultP

p.send(m, a, b) is equivalent to p.post(m, [a,b])

p.fcall(...args) -> resultP

p.fcall(a, b) is equivalent to p.post(void 0, [a,b])

p.put(name, value) -> voidP

Eventually set the value of the name property of target to value. Return a promise-for-undefined, used for indicating completion.

p.delete(name) -> trueP

Eventually delete the name property of target. Returns a promise for the boolean result.

p.then(success, opt_failure) -> resultP

Registers functions success and opt_failure to be called back in a later turn once target is resolved. If fulfilled, call success(resolution). Else if rejected, call opt_failure(reason). Return a promise for the callback’s result.

p.end()

If p resolves to rejected, log the reason to wherever uncaught exceptions go on this platform, e.g., onerror(reason).

We can compose Q.all, then, and destructuring to delay until several operands are revealed

var sumP = Q.all([xP, yP]).then(([x, y]) => (x + y);

Q.join

Join is our eventual equality operation. Any messages sent on the join of xP and yP are only delivered if xP and yP both reveal the same target, in which case these messages are eventually delivered to that target and this joined promise itself eventually becomes fulfilled to designate that target. Otherwise, all these messages are discarded with the usual rejected promise contagion.

Q.memoize

Q.memoize of a one argument function returns a new similar one argument function, except that it (eventually) calls the original function no more than once for each such argument. The memo function immediately returns a promise for what the original function will eventually return. Equivalence of arguments is determined by the optional memoMap passed in, which defaults to a new WeakMap() if absent. (Passing a memoMap in also allows the caller to seed the mapping with some prior associations.)

The difference from a traditional synchronous memoizer is that the original function is called eventually after a promise for its result is already memoized, enabling cycles to work. For example, if memoF === memoize(f) and f(x) calls memoF(x), then an outer call to memoF(x) schedules an eventual call to f(x) which makes an inner call to memoF(x). Both outer and inner calls to memoF(x) returns a promise for what f(x) will eventually return.

The argument string to spawn is evaluated in a new Vat spawned for that purpose. Spawn returns a promise for what that string will evaluate to. Once that promise resolves, the spawned vat is shut down.

Vat.evalLater() as Async-PGAS

The “at” statement first evaluates the PlaceExpression to a place, which is analogous to a vat. It then evaluates the Statement at that place. The statement evaluates with the lexical scope containing the “at” statement, so the locality of the values bound to these identifiers is the locality they have at that place rather than at the location containing the “at” statement. Although the argument to Vat.evalLater must be a closed expression (modulo whitelisted globals), we can get the same effect, a bit more verbosely, by passing in these bindings as an explicit eventual call to a closed function.

For example, the X10-ish program

const x = 6;
let ultimateP;
at (place){ ultimateP = x*7; }

can be expressed using aVat.evalLater() as

const x = 6;
let ultimateP = place.evalLater(x => x*7) ! (x);

Open Vat

The makeOpenVat function makes an OpenVat function. Each OpenVat function is like the Vat function, in that both make an return a new vat instance. Each OpenVat function additionally maintains a side table mapping from all incoming remote promises to the evaluation function of the open vat made by this OpenVat function. The reason we call such vats open is that, given a remote promise into such a vat and the OpenVat function that made that vat, one can thereby inject new code into that vat.

there

The there(p, ...) function is analogous to Q(p).then(...), except instead of merely relocating the execution of the callback in time till after p is resolved, it further relocates it in spacetime, to where and when p is near. Like Q(p).then(...), there immediately returns a promise for the eventual outcome. We do not likewise relocate the errback, so that we can still notify it and it can still react on the requesting side to a partition between the requestor and p‘s host.

Map-Reduce Lite

Given an initial result value, a list of potentially remote promises to elements, a closed mobile mapper function from elements to mapped result values, and an associative / commutative reducer function from pairs of references to result values to a new reference to a result value, mapReduce immediately returns a promise for the result of mapping all the elements where they live, and reducing all of these results together with the initial result value to a result.

I call this “Map-Reduce Lite” because, unlike a production map-reduce infrastructure, the following mapReduce does all reductions on the spawning machine, which is therefore a scaling bottleneck, and has none of the fault-tolerance. Here, any failure causes the overall map-reduce to fail, i.e., the returned promise becomes a rejected promise. The mapper and reduction functions are like the conventional functional programming variety, rather than the map-reduce variety which arranged for group-by keys.

AMD Loader Lite

This is a minimal Asynchronous Module Definition (AMD) Loader for a subset of the AMD API specification. In this subset, define is called with exactly two arguments, a dependencies list of module names, and a factory function with one parameter per dependency.

If module “W” depends on modules “X”, “Y”, and “Z”, then only once the promises for the “X”, “Y”, and “Z” modules have all been fulfilled will the “W” factory function be called with these module instances. The result of calling this factory function will then become the “W” module instance.

What it means to be the source for the “W” module is that fetch(”W”) will eventually return that source string. For example, a given fetch function might fetch it from https://example.com/prefix/W.js.

Since the memo mapping we need is from module names, which are strings rather than objects, we need to provide an explicit memoMap argument to Q.memoize, which should be a map that accepts strings as keys.

Although the cycle tolerance of Q.memoize is generally useful, here it hurts. Because define won’t call the factory function until all (Q.all) of the dependencies are fulfilled, any cyclic AMD dependencies cause an undetected deadlock. Still, in the naive absence of this cycle tolerance, such cyclic dependencies would have instead caused an infinite regress which is even worse. Better would be cycle detection and rejection, which we leave as an exercise for the reader.

Orleans: A Framework for Cloud Computing with video. Starts with the same general model – async messages returning promises, vats (called grains) processing the reception of such messages in sequential turns that run to completion, pipelined polymorphism between promises and far references. Adds scalability by on-demand grain replication and optimistic reconciliation.