I would like to take a description that tells me the order in which a number of functions are to be applied and use that to generate a composition of those functions. In other words I would like to do run-time function composition. Because the description comes from a running Scala program, I at least have guarantees that the types are correct.

Minor gripe: your func1 etc are methods, not functions. But you can use
them in places where functions are expected, and then the compiler will
convert (“lift”) your method of signature (A)B into a function A => B.

Since functions are objects, you can save them in a collection and retrieve
them at runtime according to inputs. The question is what about the static
types. There are different scenarios and approaches.

(1) You know in advance that a certain piece of code will only ever deal
with functions of a certain type, say Int => Double, so you can supply the
static types. You could have a collection of Int => Double, such as
Map[String, Int => Double], or your own collection that has a
getIntToDouble(name: String): Int => Double method.

(2) You don’t know the types of your function and you don’t care. You could
wrap your A => B function into an Any => Any function and then have a
collection of Any => Any functions.

(3) You only have a limited number of types, somehow store the type
information, and branch out. For example, you may have only Int and Double
values, then you only have four types of functions: Int => Int, Int =>
Double, Double => Int, Double => Double. You’d have to separately store the
type information to know the correct branch. This is repetitive and gets
quickly unwieldy when new types are added.

(4) You call the apply method of each function through reflection. This
circumvents any static typing.

(5) You create your composition by having your app generate the needed
Scala code, invoke the compiler to compile it, then have the classloader
load the resulting class and then you instantiate it via reflection.

The general technique for doing this is type-aligned data structures. The downside is that you cannot use the built-in collections directly for this; you must design a structure that fits your needs.

You can preserve all the type information in an hlist, but all you are doing that way is shifting the problem, if I’m interpreting your explanation correctly. (Of course, if you can use the singleton types of those strings instead of their runtime values, then there is no problem there; you can do all the lookups at compile-time. I’m not going to explain that because there’s plenty written elsewhere about that.)

Scalaz 8 includes some type-aligned structures that are commonly needed. For example, c2 can be modeled directly like so (tried in Scala 2.12.4, Scalaz 8 fcd2d2b3):

The type of this structure works by throwing away type information that you won’t need and keeping type information that you do. In this case, only the argument type Int of the first function, and the return type Boolean of the last function, are preserved. The presumption is that you don’t care about the types in between; indeed, you can’t usefully take sections of this list because you wouldn’t know the argument type and/or return type anymore. You can think of the element types as follows:

So here is the trouble with your string-selection idea. How do you limit the selection of functions to one that makes sense, i.e. one that takes an Int and returns a Boolean? You want to be able to pick func1 and func3 or all three, or func2 multiple times, but no other combination makes sense.

I don’t know exactly your goals, but here is an example of letting a particular dynamic selection goal design the datatype for me. So, like AList, I’m fixing the input and output types to T and R, but at each stage, you get to pick a “branch”, which leads either to another choice or a terminator, indicating you’ve reached the goal type.

Appreciate the feedback. In regards to options (1) and (2), they are not viable. I simply cannot foresee what functions I will need nor what their signatures will. Option (2) may be the simplest solution since I have some guarantees that the sequence of calls are valid (type checked, of-course this is true only if i do not change the definitions and recompile them). Hmmm… option (5) seems like overkill. That would be a last resort.

Stephen suggestions seem to be a more disciplined way of doing what I need, however I will do a test using option (2) to see what gives.

Some additional background. I already have a data structure that describes the functions and their composition. I use this to generate a set of composed functions using partial evaluation. Each composition is a “pipe” that consumes data and produces results.

S11001001:

The presumption is that you don’t care about the types in between

This is correct. In fact both the AList1 and the ResPair have the same “intermediate” type that I use.

S11001001:

But what if you passed [a, b] => a => Either[Result[A], b]?

Interesting. Although in my case I don’t need this because the possible “pipes” are type checked at compile time. At run-time partial evaluation constructs the “pipes” (function composition).

S11001001:

I don’t know exactly your goals,

Once the “pipes” have been constructed they are executed. During execution each function and the parameters used in the pipe are recorded. I want to take this recording and simply recreate a single “pipe” for use with another input.

S11001001:

but here is an example of letting a particular dynamic selection goal design the datatype for me.

Wow. This has given me an idea. I may adapt my existing “pipe compiler” to traverse the “pipe structure” again but now, using the list of function names, select and compose a single “pipe” much as you do in your interpreter (the parameters my be an issue though).

Hmmm… now this leave me wondering if I can “pickle” the “pipes description” to a file.

Your example also shows me that we can “delay” function composition to run-time and still ensure type safety. Still have to wrap my head around this.