Saturday, February 02, 2008

Continuations: Wistfully yearning

During the past week, there was a conference at Microsoft's campus in Redmond, Lang.NET. I didn't attend, but I did look over again at the videos from the previous speakers (one of which was by Danny Thorpe).

However, it wasn't Danny's talk that I found most interesting. Instead, it was Shriram Krishnamurthi's presentation on continuation-based web application programming.
Ian Griffiths wrote a disparaging post on continuations describing how they can work in a web application, in case you're not aware of their nature. I strongly disagree with Ian, though, to the point that I'm not actually sure if he's willfully misrepresenting those who would advocate continuations.

In my previous job, we developed a web application framework, running on top of ASP.NET, for data-entry intensive applications in the finance industry. The product is now called Topoix. I contributed a bunch of things to the underlying architecture and implementation, but one of them is particularly relevant to continuations: the way Topoix implements modal dialog boxes that ask the user a question. Essentially, displaying a dialog caused execution of business code on the server to stop and resume correctly, based on the reply, when the dialog was dismissed. The stopping and resuming didn't tie up a thread on the server, and nor did it require stickiness to a particular server in the farm, but obviously it required some state - persisted to disk and perhaps cached in memory, depending on how you wanted to configure things.

Now, I want to address some of the (IMHO) mistaken criticisms in Ian's article.

This requires a certain amount of magic under the covers. Supporting continuations in special cases such as function calls or iterators is much easier than providing completely generalised support. Continuations do not fit all that well with the stack-oriented execution model offered by the JVM or CLR.

I agree to the point that code that hasn't been transformed into continuation-passing style doesn't fit well with a stack-oriented execution model. But, however, bear in mind that translation from stack-orientation to continuation-passing style is a safe and semantics-preserving operation. Things really only get hairy if one uses callbacks from non-managed code, and one expects to be able to pass around a continuation with the non-managed code on the stack.

Ian continues:

Continuations can look attractive on the web, because they offer a tool that lets you capture the shape of a sequential user journey in the structure of your code.

Continuations can do more than capture sequential user journeys; if you take care to capture values, rather than references, they can capture an arbitrary history tree of independent, concurrent user journeys. This is because the stack in CPS is essentially converted into a linked list of activation records in the heap, where each activation record points to its parent. It's trivial to see that continuing at an earlier point (equivalent to pressing the back button in your browser) simply creates a new activation record which points to some tail of the linked list, and doesn't disrupt the concurrent "main trunk" of interaction. Providing that you don't mutate variables from previous activation records (unless that's desirable - consider e.g. site preferences, or login status), you don't have a problem with non-sequential navigation.

However, I think this is a bad idea. Although the relationship between the code and the user navigation path is apparently simple, it hides subtle but significant details. This makes the same mistake as we did in the 1990s with distributed object models. We’d like to believe we’re hiding a lot of implementation details behind a simple abstraction. In practice we’re hiding important features behind an inappropriate abstraction. Our industry has more or less learned the lesson that procedure calls are an unworkably naive abstraction for request/response messaging over a network. I suspect that it’s equally naive to attempt to manage the complexities of user/web server interactions with simple sequential code.

Ian here is saying that bad abstractions are a bad idea. I fully concur; when building abstractions, it's important to abstract the correct things, and not encourage writing the wrong sorts of programs. There are a few things to say about this, though. The first is that the really egregious abstractions are those which enable writing programs that one would not actually *want* to write, if one fully understood the abstractions in hand. I'm thinking in particular here of cross-machine COM or CORBA: we should prefer REST or document-style SOAP for reasons that are beyond the scope of this post, but I'll just say that they relate to the nature of object orientation as used in the small, with shared memory, on local machines, and how that falls apart when distributed.

However, I think Ian will have to come up with something better than condemning continuations as only being capable of abstracting "simple sequential code", if for no other reason than it's a falsehood.

So, Ian tries to come up with other reasons to slay the continuation beast. However, it's worth bearing in mind, when considering his objections, what the alternative is, and then asking one's self if the alternative isn't actually a continuation written out the long way.

Let's see.

Abandoned Sessions. Sometimes the user just walks away.

Well, a continuation that one persists to a cookie, hidden form fields, encoded in the URL, or to server-side storage, as necessary / appropriate, puts this one to rest. And besides, how does one solve this issue in the average trivial shopping-cart site? One must save state in exactly the same locations, but you have to encode the resumption of logic yourself, rather than having it done for you. The abstraction doesn't look weak here.

Now, there are things Ian says that aren't related to continuations per se, but rather to coding a web application as if it had state.

Continuation-based web servers aren't a way of pretending that the server is stateful when it needs to be stateless. Rather, they're a way of easing the pain of getting back to where you left off. That's why, when Ian says:

The problem with this is that a lot of the techniques we have learned for resource management stop working. Resource cleanup code may never execute because the function is abandoned mid-flow.

... it's not very relevant. One can't be holding any non-persistable resources when persisting a continuation, and with language and library support, this can be flagged at runtime or ideally at compile time (I can't help it, I'm a static guy, even though it's not fashionable in the web world). And since the continuation-persisting code needs to, you know, actually persist the objects on the linked-list stack, it can trivially flag the obvious problems when it discovers something it can't persist.

Thread Affinity. With an ordinary sequentially executing function, I can safely assume one thread will run the function from start to finish. But if I’m using continuations to provide the illusion that I’ve got sequential execution spanning multiple user interactions, then I might get a thread switch every time I generate a web page.

I see this as kind of a bogus problem. How often, when writing an ASP.NET application, does one work with objects or resources that have thread affinity, and expect them to be on the same thread when you next enter the server? I should think one doesn't expect this at all - one doesn't normally have the ability, much less the "problem", of handing an object from one request/response cycle to the next.

Web Farms. This is essentially the same problem as the thread affinity issue, but at the machine level: in a web farm, your sequential function might end up executing on a variety of machines over its lifetime. However, you’d probably avoid this problem in practice using sticky sessions. (And unless your continuations are serializable across machine boundaries you’ll have to do this.)

I don't see any point to using continuations if they can't be saved and resumed at will, across process boundaries, machine boundaries, server restarts, etc. If you limit the power of the continuations you consider, it's pretty easy to condemn them as not being powerful enough.

The Back Button and Branching

This one’s the killer.

Your web site may present linear user journeys, but that doesn’t mean your users necessarily follow them. I often don’t.

I habitually do two things that will confound any web site that expects the user to do things in a particular order. First, sometimes I use the back button. Second, sometimes I bifurcate my navigation - I’ll open a link in another tab. Both of these will confuse any web site that thinks it knows what my ‘current page’ is. The notion of a current page is not enshrined in either HTTP or HTML, and I enjoy the flexibility this offers me when browsing sites. Indeed, it’s one of the reasons I really like tabbed web browsers.

Here, I don't know what Ian is talking about. I mean, I understand exactly what he's saying, but it's a complete non-sequitur to jump from this behaviour to saying that it's incompatible with continuations. Continuation state forms a linked list on the heap. One can share the tail of a linked list among an arbitrary number of heads with zero problems.

This means I have to write my function in such a way that it can cope not only being rewound, but also to being split so that multiple threads execute the function simultaneously, each taking different paths. But of course because I’m using continuations, each of these threads gets to use the same set of local variables. The fact that I enabled users to inject gotos into my code at will is now looking like a walk in the park - now they can add arbitrary concurrency!

Here, when Ian says "use the same set of local variables", he's talking about mutating the tail of the list, which might be shared by other heads. And he's right in one sense: if you modify the tail, you better mean it, because you're modifying shared state, not local state.

However, I have a couple of more aces up my sleeve. If desired, the transformation that turns sequential code into CPS can also turn mutating assignments into single static assignments. Single static assignment is a form used in some compilers in the back end to ensure that a variable can only ever have a single definition. If one could annotate one's variables, indicating if they should be shared across multiple continuations (stored in the tail) or strictly local to every continuation that redefines it (essentially,
hidden in a scoping sense by the redefining activation record), this problem goes away.

The deeper objection I have to Ian's criticism is that you have to deal with this problem anyway, and program transformation just makes your life easier. Consider what you'd have to do to handle Back Button and Branching without using the approach sketched above. You have all the same problems. You still need to make sure that wherever you stuff away your state, that it isn't inappropriately used by the branches. SSA transformation can do this automatically and declaratively.

Consequently, this approach requires you to write your code in such a way that it can tolerate sudden halts, thread switches, rewinding, and forking of execution.

Sudden halts, thread switching, rewinding, forking: what is it in the tear-down and re-setup of the ASP.NET page cycle that stops you from seeing these things, if you choose to describe them that way? Tearing down the page is surely a violent halt. Setting up again, possibly on another machine, is surely a wrenching thread switch away from your carefully constructed state. To support rewinding / forking (aka Back button / new tab / window), you still need need to juggle your state so that it's correctly stored, keyed and associated with its page which was originally displayed to the user. But continuations can do *all* of these things with easier to read code, and selective SSA transformation turns a fiddly choice (should this data be in the database, in memory, in a cookie, querystring, hidden field, client-side state sent back through AJAX) into a declarative form which your Web library should be abstracting for you.

I will give Ian something, though. It's certainly not true (and shouldn't be true) that one simply writes a sequential application using continuations without thought for the network and then expect it to work fine across the web. That can't happen. However, I do believe that with the correct libraries, language, annotations and transformations, a far better language for web apps can be created. Whether it'll be along the lines of Volta, or one of the frameworks described herein, I don't know. However, I strongly suspect new language constructs, or at least flexible DSL support, is necessary. I can even think of implementing continuations using monads, so perhaps it'll be Haskell!

1 comment:

A node in a linked list presents a linear view. Navigation through the web needn't be linear, and frequently isn't. I know you have responded to this, but I think you've missed an important point. You say that continuations "can capture an arbitrary history tree of independent, concurrent user journeys" but while that's true, the tree is not visible in tree form to code running in the web application. The tree does not form part of the programming model: the web application only ever sees one path through the tree - it only ever sees lines. It may see different lines at different times, but the model it presents is always linear.

What, you may ask, of the fact that it sees different lines at different times? Yes, a tail may be shared by multiple heads, but looking at the view available from the point of view of any one particular node, it looks linear - linked lists don't give me a way to join multiple paths back together. The tree may exist, but my code never sees it as a tree, because the continuation model always shows me the view from a particular node. I never see the tree as a tree. This matters.

One practical upshot (not the only one) is that the tree is as good as pruned the moment the user has no way of getting back to a particular path they abandoned. E.g.:

That path that started on step 3 - where is it now? It's probably still a leaf in a tree on the web server at this point, but because the browser will have pruned that branch from its history, the corresponding branch on the server will never be seen again, unless you get a core dump and go looking for it with a debugger, or go ferreting around in your continuation persistance store. As far as the code running on your web server is concerned, the path started in step 3 is gone for good.

The code that ran to serve up page D on step 5 will not see evidence of that click to page 3. At least, if you've used single static assignment as you suggest there will surely be no evidence at all - any work done in 3 will be stored in come continuation that there's no longer any way of getting to. The user's browser will have discarded that path as soon as step 5 occurred, so even though your server may be maintaining a tree structure that still has that continuation in some form, the browser has just discarded the information that would have been required to get to it.

So unless your server has some mechanism to get the work done in step 3 back onto the branch we entered on step 5, that work is gone for good, along with any data the user may have supplied on that step.

Of course, you could allow mutation of state shared between continuations, on languages that support this, and that would provide a way for side effects in step 3 to be visible to step 5. But I hope you would agree that madness that way lies - you proposed single static assignment, and speaking as (amongst other things) a Haskell developer, I wholeheartedly agree with that as a strategy. But you now need some other mechanism (e.g. state in the database) if you want work done in step 3 not to be lost.

That's what I mean when I say that I don't think continuations deal with non-linear navigation. Their 'solution' to non-linear navigation is to act as though none of the forks ever happened.

On that point, you spoke about the "main trunk" of interaction. This is a telling phrase. It reveals a presumption of an underlying linearity, and this illustrates exactly what I'm talking about. You suppose that there is a 'main trunk' of interaction. This supposition is at the heart of continuation based frameworks. Unfortunately, this supposition is wrong.

That's the fundamental problem. That's why I identified it as "the killer." Apparently I've not explained myself well - rather than directly disagreeing with this point you said that you "don't know what Ian is talking about". So I'm going to try and explain this one point a little better.

Much as I want to reply point by point (and please email me if you want to follow this up in more detail), I fear that the key point will be lost in the detail, just as it apparently was with the original post. With hindsight I wish I hadn't bothered with most of the other stuff - I believe it's real and more relevant than you give it credit for, but it's prosaic stuff, and not really at the heart of the problem.

The heart of the problem is that web navigation is not linear. As far as I can tell (and your post, with its presumption of a "main branch" seems to reinforce this) continuation-based frameworks present web navigation to the developer as though it is linear. Indeed, most web continuation advocates seem to think that's a positive feature, perhaps even the point. I think it's a problem.

Pretending that a linear view of a user's navigation can accurately represent their work will hurt the user more than it hurts the developer. That's a problem if you're a user.

"Make the easy case easy, and the hard case ignorable; screw the users" is, I hope, not a motto any developer would aspire to. And yet this seems to be the upshot of continuation-based web frameworks, if what you say here is correct.

Here's a simple concrete example. Suppose I'm on Amazon, and I've decided to buy a book. I click the necessary links to add it to my basket. Then I decide I'm going to buy another one that I saw earlier, and for whatever reason it's not one of the ones Amazon has decided to show on the "why not buy these too" type page. So I just open the Back popup to skip back a few pages and find the book I wanted. Then I go through the purchase process again.

My navigation path wasn't linear here. It's forked. If a linked list truly is a good representation, I'm now hosed. Sure, a tail can be shared amongst an arbitrary number of heads, I want those forks rejoined: when I got to check out, I want *both* the paths I took from the branch to be taken into account. I.e., I want to buy both books.

(And in practice, whenever I'm on Amazon, I usually end up with many tabs open so I can keep track of several books at once. So it's rarely anything like as simple as this single-branch example I've just presented.)

The ability to have a tail with two heads is necessary but it's not sufficient. I also need a head (the final checkout) with two tails (the two different paths I took from the point of splitting in which I wanted two different books). For that, you'll need something other than a linked list.

The way continuation-based web sites have thus far been explained to me by their advocates seems to indicate that continuations are no good for this scenario. I had a conversation at JAOO in 2006 with a guy who's spent a lot of time working with Seaside about this exact scenario, and his response was essentially that you had to use some other mechanism to manage the join here.

Well OK - solutions exists. But I'm now confused because I have to look outside of continuations if I want to solve problems with branching and the back button. But isn't that exactly what continuations were claiming to do for me?

Worse, I could easily ignore the problem. Continuations handle one case here perfectly: if I go back, and fork in a way where I really do want to abandon the original branch, I'm good. If there really is, as you say, a "main trunk", and if you're actually happy that other branches don't, as you put it "disrupt" that trunk, then that's great. Continuations let me act like nothing outside of the "main trunk" ever happened. I can imagine that anything my user did outside of this "main trunk" is unimportant. I get to pretend that my world is nice and linear, like a linked list.

But if I care about the abandoned branch, as I will if anything remotely interesting happens on it, I now have a programming model that's actively dangerous. If I ask the question "how did I get here?" the programming model tells me "in a straight line" whether that's true or not. Speaking as a developer, I don't want to work with a mendacious programming system. More importantly, speaking as a user, your application lost the data that I entered on the branch you decided to ignore.

Let me just say that again.

YOU LOST MY DATA.

Users hate that. I believe that developers should hate that too. In practice, a scant regard for the worth of the user's time and data seems to be the norm on the web. This depresses me. It's one of the reasons I wanted to speak out on this topic - it seems like another example of prioritising developers' convenience over users' time and data.

It seems that the continuation-based approach is only safe in applications where it's invariably OK for you to forget that the user asked you do to something. That doesn't sound like a hugely useful set of scenarios to me. So in practice, if you care about your users you're going to have to be mindful of the possibility that the picture your programming system presents to you reflects only one slice of a larger reality.

You say:

"The deeper objection I have to Ian's criticism is that you have to deal with this problem anyway"

Interestingly, that's precisely *my* objection to the continuation solution. You have to deal with joins anyway. (If you care about your users, that is.) Continuations make that harder. Transforming your program in a manner that makes this harder doesn't seem like a good way to make your life easier.

By the way, I would like to point out that my original point was nothing stronger than that I believe the approach has serious shortcomings. I said a lot to back that up, but I don't think the conclusion was so very wild. And in my followup later that month, I clarified what I felt the issues were: 1) it makes certain classes of bug much easier to introduce and much harder to spot; 2) execution of code is a very bad abstraction for user journeys because these two things have completely different shapes. It would be a mistake to extrapolate far beyond that. I do not argue, for example, that these problems necessarily mean there is no place for continuations in a web based framework. With a full awareness of the pitfalls, it may well be possible to design frameworks in a way that guards against these problems and still allow worthwhile use of continuations. I still believe continuations are far from a complete solution to navigation structure, and that they introduce new problems of their own in this space. But all technologies bring new problems. The key thing is to acknowledge and fully understand the problems - only then will we able to guard against those problems. A large part of my motivation for those original two posts was that I felt everyone talking about this technique was enamoured with the cleverness and blind to the flaw.