Wednesday, May 27, 2009

Optional laziness doesn't quite cut it

Strict languages that support optional laziness just don't provide the same modularity and composability that you get with a language that is lazy by default. In a strict language with optional laziness, callers (direct and indirect) all may need to be aware of the laziness of any potential callee (direct or indirect). This is a devastating handicap that makes these functions much less useful and composable than they would be otherwise.

Here's an example, in Scala. Suppose we create a method, foo, lazy in its second argument:

def foo[A,B](a: A, b: => B): B = ...

If we want to call foo, no problem, just do foo(a, hugeFunction()). Scala automatically creates a thunk wrapping the hugeFunction() call, ensuring that it won't be executed before foo gets called. The problem comes when we consider a caller of foo:

This isn't quite right - bar is strict in its second argument, but when that argument is actually used, in the call to foo, it is lazy. If foo depends on the laziness of b for its semantics or performance, we probably should adjust the signature of bar to also be lazy:

def bar[A,B,C](a: A, b: => B, c: C): B = ...

Starting to see a problem yet? What about callers of bar? These need fixing, too. In principle we might have to adjust the signatures of a long chain of callers into foo, until we get to the function that generates the instance of B that gets passed ultimately to foo. The problem is especially bad when writing generic code that can appear in several locations in the call graph. Very often these functions will not have full knowledge of who they call; the only way for them to be fully general is for them to be lazy by default (since some callee may depend on this laziness). If you program in a style where you make plentiful use of HOFs, or you make use of a lot of generic interfaces, you start to see what a PITA it can be to deal with strictness. A function that appears deep in the call graph needing to be made lazy might result in the programmer having to propagate laziness information to a huge number of dependent functions.

As an example, I recently built an experimental combinator library for defining parallel computations. I won't go into details here, but you can build a nice set of combinators all derived from just three primitives: unit, map, and join. As it turns out, these three functions form a monad, and the combinator library I was going to write was just going to be the usual combinators that can be derived given the monad interface. I figured I'd just save myself the trouble and just use implementation of the monad combinators from scalaz.

... except, the monad interface in scalaz defined unit to be a strict function (this is being fixed in an upcoming release), and my particular implementation of the monad interface needed unit to be lazy. Short of modifying scalaz, my only option was to rewrite all the functions that called into unit. Unfortunately, the unit function has a lot of dependents - the functions that can be derived given a monad have a nice, layered structure, where each layer builds on the layer below, with the bottom layer being just the three functions that define a monad. Changing the signature of unit in the interface causes a cascade of changes to all the layers above.

There are certainly some problems with laziness being the default, but I've started to think the problems are of the type that can be solved or have already been solved by better technology (I'll talk more about this in a future post). In contrast, the above problems with strict-by-default are fundamental.

Is it necessary to make things as complicated as Scala's syntax? I mean, Scheme has "delay" special form which also provides lazy evaluation (From what I hear, ML also provides roughly the same thing), and I see no problem in this approach.

@Magice: The syntax doesn't matter - in Scheme or ML you have the same fundamental problem. The caller must implement the laziness policy of the callee in order to preserve the callee's semantics. So instead of having to implement the laziness policy in one place - the callee - you end up have to having to implement it in N places, possibly all along the call graph leading into the callee.

@Matt Hellige: I have a crazy theory that you should really never need strictness annotations in a lazy language. The problem with static strictness analysis is that a polymorphic call site will need to fall back to lazy argument passing unless the compiler can prove that all the usages are strict. This makes strictness analysis less useful than it would be, esp if you program in a style with lots of HOFs and hence lots of polymorphic calls. I think there is a way of addressing this with runtime checks - there is some cost to doing it, but it might be less than the cost of having to heap allocate thunks unnecessarily. More on this in a later post.

@Paul: You mentioned that you have this "crazy theory that you should really never need strictness annotations in a lazy language".

I don't quite agree. In Haskell, for example, in practice a lot of functions turn out to be "almost strict" but not quite. Artificially making them really strict (by means of seq and the like) then aids the strictness analyser and, hence, triggers further optimisation of your code.

Indeed, I think seq has proved itself valuable to Haskell programmers time and time again. It is important to realise that seq (or, more general, any facility for strictness annotations) isn't there because strictness analysers are inadequate; it's there to enable programmers to more often pick the fruits that automatic strictness analysis and the optimisations driven by it offer.

@Stefan Holdermans: I think that often, manual strictness annotations ARE workarounds for deficiencies in Haskell's SA - think functions like foldl'. But I would not be surprised if this weren't always the case. Like I said, this is a "crazy theory" of mine that might not survive close scrutiny! Could you give an example where you think the manual annotation isn't something the compiler or runtime could do automatically?

@Stefan: Ah, I suppose I can't argue with your example! :) You've convinced me! There will always be SOME cases where programmer provided annotations are necessary. Another good example might be something like a tree insertion - the programmer might prefer that the insertion finish completely rather than have it be lazily pushed down into the tree on lookup.

I think where we may disagree is on how often that should be necessary in typical programming. Even in your example, since f x y is strict in its first parameter, after we pass that parameter there is enough information to determine if the second parameter could also be made strict (since at this point we will know which branch of the case we are in). The problem with a static analysis is that it forces all the branches pay the cost of laziness, when in fact only one branch requires it.

The real issue here is that evaluation is a side-effect. In Haskell, strictness isn't captured in the type which means Haskell is polymorphic over evaluation. I think this polymorphism is far more important than whatever the default happens to be. With polymorphism over strictness, strictness-analysis can propagate the side-effect based on usage, thus freeing the user from making annotations everywhere. This is much the same as type inference removing the need for most type annotations.

And with a decent compiler behind the strictness-analyzer, we can automatically perform lifting to move strictness requirements into the caller or further up the call stack. That is, rather than having a "call" be defined as evaluating (or not) all arguments before jumping, we can dissociate the jump from the evaluation thus allowing the evaluation associated with a call to be hoisted out to the caller's annotations (or beyond). Of course, doing this requires the ability to track other side-effects in order to ensure they aren't reordered inappropriately.

@wren - I am skeptical that static strictness analysis of any sort will ever be sufficient, although it can get you part of the way there. Even after adding strictness information to the types, you'll still have to be unnecessarily conservative when strictness properties depend on the runtime *values* of arguments passed.

A contrived example:

f x y = case x of 1 -> strictFn y 2 -> lazyFn y

To go the last mile, I really think you need something like a runtime that specializes functions as arguments are supplied. I've been thinking more about this lately... I'll try to post some of my ideas soon.

I think wren has hit the nail on the head here. Evaluation is a side-effect, and once a value has been evaluated, that can't be undone.

This makes the debate over proper polarity unsymmetrical:

Both haskell and scala use type annotations but the difference (as you show) is that in scala you need every annotation up the call stack to be lazy in order for the default evaluation to be overridden. While, in haskell, you need just one annotation anywhere in the call stack to override default evaluation strategy.

Since evaluation is a side-effect, it might be interesting to explore forcing thunks explicitly; the type of unevaluated B and evaluated B would be the same. When the value of B was needed or you deduce it's time to evaluate it, do so explicitly. (presumably all primitives would do this implicitly). This sounds like what you want, right?

Alas, I think that leads back to default laziness, and required strictness analysis to get back performance.

Because all primitives need values, not thunks, you now need to insert (force foo) around all primitives unless you can prove that the values have been evaluated already. That sounds like the dual of strictness analysis.

But, 1) the "is evaluated" analysis might be much easier than strictness, and 2) an unnecessary force is just a missed branch prediction, instead of a potentially unbounded computation.

Your example is striking in that it is *exactly* the same example that is brought against monads as ways of controlling effects. If the inner function suddenly decide to use a monad, it changes its type, so you must change the caller, and the caller's caller, etc. "This is not modular", blah blah blah.

Is that a defect of monads or something fundamental about effects? How do we solve this kind of problem of "callstack creep"? I don't know. Still I'm wondering why you didn't mention that it is the exact same problem.

Indeed, lazyness has a monadic structure, so your example is actually a special case of the general example against monads. If we try to generalize your conclusion/solution as well, we get: "drop effect annotation in types, let's live in a non-pure world were effects can be used silently, it's more modular". What do you think about this reasoning?

I have no solution, no answer to the riddle. Maybe lazyness is special and your solution is actually "the right one" in this restricted case. But I doubt it; I think there is something about the interaction of types and modularity that we don't quite understand, or have not formulated clearly enough.

@gasche - I don't think strictness vs laziness is analogous to the situation with monads in general. In general, converting some portion of pure code to monadic code is not trivial - there can be many ways to go about it. In addition to typing issues (if the monadic effect is erased from the type, we have issues with mixing of monads not being caught by typechecker), the programmer must make decisions about where and how they are injecting the effects, how they are ordering them, etc. Hence we can't just remove the effects from the types. Contrast this to laziness vs strictness, where it actually works quite well to not track strictness info in the type, default to normal order evaluation at runtime, and just let the compiler / runtime choose where to insert evaluation (and I think this can be made to work even better with smarter evaluation strategies that have the same termination properties as normal-order eval).

The way I think of this is that laziness by default lets functions be polymorphic in their strictness, deferring to the functions they receive or call for the strictness policy. This means they can be reused moreso than strict functions, which have basically hardcoded an evaluation strategy. Same goes for strict vs lazy data types.

On a somewhat related note, I am actually open to the possibility that you could have a strict-by-default language with an explicit form of strictness polymorphism in the type system to let you avoid the code duplication (I've seen some suggestions of that and Stefan even posted an embedding of this idea into Haskell on the mailing list a while back). This isn't implemented by any existing strict language I know of, but who knows, maybe it could be workable and as convenient as lazy-by-default languages.