nótaí de devos

Homage to the Transducer

Empirical learning might be characterized as repeated application of a function (S, A) => S, assuming S is existing knowledge and A is a new observation.

In clojure terms this signature is the basis of something called a reducer.

A transducer is something that transforms data that will ultimately be reduced. It is formulated as a function taking a reducer and producing a new reducer that accepts different input or treats the input differently.

Context

The type constructor Context adds nothing to the foregoing definitions because Context[State] is defined simply as State.

However, Context can be defined as any functor. For example, scala.util.Try to capture errors or scala.concurrent.Future for an asynchronous reduction.

The result of the reduction is obtained in whatever context is used, that is, reduce returns a Context[S], given a Reducer[A, S].

If Context is defined as scalaz.concurrent.Task or flowlib.Process the entire reduction is captured as a suspended, potentially asynchronous, computation which can be subsequently run or composed with others.

Reducible

A typeclass Reducible is used to make sources of data eligible for reduction.

An implementation note: Reducible instances are defined differently for each concrete Context. In theory, if Context was required to have a flatMap operation, we could consolidate these definitions. In practice, many concrete Context types don't support deeply nested flatMap calls.

Transducer

A Transducer[A, B] converts a reducer of A's to a reducer of B's. It does not depend on the result type, S.

In this example, r2 has result type Double but internally r2.State is (Double, Int). The Double tracks the state of r1 and the Int tracks the count of values seen. Ultimately, r2.complete is called which strips the count from the result.

Note that reducers and transducers are immutable. Reduction state is created and contained within the reduce operation.

Conclusion

The point of this is not to compare clojure and scala but to attempt to borrow from clojure.

I think Reducer, Reducible and Transducer have emerged as quite reasonable types in scala. Each is a polymorphic function, which is not surprising given we are coming from clojure. The types surely don't model every rule, but they say quite a bit. And the whole thing is pure (important for my concurrent programming use case).

I now want to see how useful this is in practice, particularly with type Context[S] = flowlib.Process[S].