the "contains" problem

Since I was so recently discussing with the cc: recipient the annoyance imposed by the absence of errors or warnings here:

List("a", "b") contains 5

This is a consequence of the inability to selectively "turn off" variance (without having to rig up something silly like an implicit from List[T] to MyInvariantList[T]) thus requiring contains to be unbounded on the upside, i.e. Any. It looks like they've done something sensible about this in kotlin:

"What happened here is called type projection: we said that from is not simply an array, but a restricted (projected) one: we can only call those methods that return the type parameter T, in this case it means that we can only call get()."

I propose we steal this, but only after adding some needless complexity so it fits in better with our language.

It's a neat idea. Mixing usage site variance into scala. If we bump up the complexity, I vote using smileys (☺ and ☻) for notation.
Seriously though, anything to help make variance easier to deal with would be great.

Since I was so recently discussing with the cc: recipient the annoyance imposed by the absence of errors or warnings here:

List("a", "b") contains 5

This is a consequence of the inability to selectively "turn off" variance (without having to rig up something silly like an implicit from List[T] to MyInvariantList[T]) thus requiring contains to be unbounded on the upside, i.e. Any. It looks like they've done something sensible about this in kotlin:

"What happened here is called type projection: we said that from is not simply an array, but a restricted (projected) one: we can only call those methods that return the type parameter T, in this case it means that we can only call get()."

I propose we steal this, but only after adding some needless complexity so it fits in better with our language.

List[A] would then be invariant and have an implementation of
Covariant[List] and the contains method would also accept an
implicit argument of type Equal[A], which itself, has an
implementation of Contravariant[Equal].

It's a neat idea. Mixing usage site variance into scala. If we bump up the complexity, I vote using smileys (☺ and ☻) for notation.
Seriously though, anything to help make variance easier to deal with would be great.

On Wed, Jul 20, 2011 at 9:24 AM, martin odersky wrote:
> On Wed, Jul 20, 2011 at 5:58 AM, Josh Suereth
> wrote:
>>
>> It's a neat idea. Mixing usage site variance into scala. If we bump
>> up the complexity, I vote using smileys (☺ and ☻) for notation.
>> Seriously though, anything to help make variance easier to deal with would
>> be great.
>>
> How is this different from wildcards/existential types?

Can you please elaborate how "contains" will look like with this approach?

I think this is a different problem: we'd like to use a covariant type parameter in a contravariant position
As far as I understand (is there a more detailed spec?), Kotlin's type projections can only be used to turn invariant type params into co/contravariant ones, by blocking access to members that mention the type param in a contra/covariant position.
Since the type of the target of a method selection can be subsumed, a List[String] can be re-interpreted as a List[Any], and must thus support the same operations. In theory, if you're certain (statically) that you know the exact (dynamic) type of a target of a method call, you need not worry about this and could allow a covariant type parameter in a contravariant position. However, Scala does not have this notion of exact types.
On the other hand, implicit conversions do not perform this subsumption, so you can do something like:scala> case class Source[+T](val contents: T)scala> trait Contains[T] {
| def contents: T | def contains(x: T) = contents == x | }scala> implicit def containsSrc[T](s: Source[T]) = new Contains[T]{def contents = s.contents}
scala> Source(1) contains "1"<console>:12: error: type mismatch; found : java.lang.String("1") required: Int Source(1) contains "1"
^scala> Source(1) contains 1res1: Boolean = truescala> val x: Source[Any] = Source(1) // we have to make subsumption explicit
x: Source[Any] = Source(1)scala> x contains "1"res2: Boolean = false

Can you please elaborate how "contains" will look like with this approach?

I think this is a different problem: we'd like to use a covariant type parameter in a contravariant position
As far as I understand (is there a more detailed spec?), Kotlin's type projections can only be used to turn invariant type params into co/contravariant ones, by blocking access to members that mention the type param in a contra/covariant position.

That was precisely the idea of Viroli and Igarashi. Their underlying formalism were existential types. Their work led to Java wildcards, which replaced existentials by something which is a bit easier to use but far more hairy to spec & understand. I still claim that the number of people who really understand wildcards number fewer than the fingers of one hand. I wonder where in that spectrum Kotlin is.

On Wed, Jul 20, 2011 at 10:14 AM, Paul Phillips wrote:
> This is a consequence of the inability to selectively "turn off" variance (without having to rig up something silly like an implicit from List[T] to MyInvariantList[T]) thus requiring contains to be unbounded on the upside, i.e. Any. It looks like they've done something sensible about this in kotlin:
>
> http://confluence.jetbrains.net/display/Kotlin/Generics#Generics-Typeprojections
>
> It says:
>
> "What happened here is called type projection: we said that from is not simply an array, but a restricted (projected) one: we can only call those methods that return the type parameter T, in this case it means that we can only call get()."
>
> I propose we steal this, but only after adding some needless complexity so it fits in better with our language.

What emerged was that Set combines two roles
(a) a contravariant predicate that tests for membership of the Set
(b) a covariant container that can enumerate the members of the set

In that discussion, after some silly stuff, I eventually said
something sensible: "if there are 2 representations of
the Set concept, that differ computationally, doesn't that suggest
they are best modeled via 3 types:

- a covariant SetContainer, being a collection without duplicates.
contains(Any) defined by enumerating its elements.
- a contravariant SetPredicate (membership test)
- the current invariant Set (collection plus membership test defined over T)"

I still hold that view. The problem is, such as change to a key type
like Set may be/appear too disruptive to pursue or win support for.

It is exactly these kinds of "legacy API" use cases where it might be
nice to give "wiggle-room" in variance to the _client_ of an API, so
that they can narrow existing APIs, like Set, down to the operations
they use.

For example, if one uses a Set object only as a container, treat is an
being covariant. If one uses it purely as a predicate, then allow it
to be contra-variant.

In summary: definition-site variance would be Scala's default.
Constrained use-site variance an optional override, where a client has
different, probably narrower, usage patterns for an API than the API's
designer anticipated.

On Wed, Jul 20, 2011 at 10:14 AM, Paul Phillips <paulp [at] improving [dot] org> wrote:
> This is a consequence of the inability to selectively "turn off" variance (without having to rig up something silly like an implicit from List[T] to MyInvariantList[T]) thus requiring contains to be unbounded on the upside, i.e. Any. It looks like they've done something sensible about this in kotlin:
>
> http://confluence.jetbrains.net/display/Kotlin/Generics#Generics-Typeprojections
>
> It says:
>
> "What happened here is called type projection: we said that from is not simply an array, but a restricted (projected) one: we can only call those methods that return the type parameter T, in this case it means that we can only call get()."
>
> I propose we steal this, but only after adding some needless complexity so it fits in better with our language.

What emerged was that Set combines two roles
(a) a contravariant predicate that tests for membership of the Set
(b) a covariant container that can enumerate the members of the set

In that discussion, after some silly stuff, I eventually said
something sensible: "if there are 2 representations of
the Set concept, that differ computationally, doesn't that suggest
they are best modeled via 3 types:

- a covariant SetContainer, being a collection without duplicates.
contains(Any) defined by enumerating its elements.
- a contravariant SetPredicate (membership test)
- the current invariant Set (collection plus membership test defined over T)"

I still hold that view. The problem is, such as change to a key type
like Set may be/appear too disruptive to pursue or win support for.

It is exactly these kinds of "legacy API" use cases where it might be
nice to give "wiggle-room" in variance to the _client_ of an API, so
that they can narrow existing APIs, like Set, down to the operations
they use.

For example, if one uses a Set object only as a container, treat is an
being covariant. If one uses it purely as a predicate, then allow it
to be contra-variant.

In summary: definition-site variance would be Scala's default.
Constrained use-site variance an optional override, where a client has
different, probably narrower, usage patterns for an API than the API's
designer anticipated.

-Ben

I've been experimenting with the following pattern for a while to get around variance issues. I know, it is hackish around the corner and its been only a few weeks, but so far I find it very easy to work with and I find the explicit coercion adds some clues to the code. Maybe it is time for feedback :)