This seems better suited to implementing as a compiler plugin than in scalac itself.

Nate

On Wednesday, June 29, 2011 at 14:56 , Branks wrote:

> Hi,
>
> Just wondering whether or not it would be good to have an "immutable"
> compiler option added to scalac. For example:
>
> scalac -Ximmutable
>
> Any scala source that makes use of vars or immutable types would fail
> to compile. This could also be a good tool for those wanting to
> either enforce or learn the functional style.
>
> Branko

Has anybody written one that does this?
Is there a public repository of shared compiler plugins out there?

On Jun 29, 11:09 pm, Nate Nystrom wrote:
> This seems better suited to implementing as a compiler plugin than in scalac itself.
>
> Nate
>
>
>
>
>
>
>
> On Wednesday, June 29, 2011 at 14:56 , Branks wrote:
> > Hi,
>
> > Just wondering whether or not it would be good to have an "immutable"
> > compiler option added to scalac. For example:
>
> > scalac -Ximmutable
>
> > Any scala source that makes use of vars or immutable types would fail
> > to compile. This could also be a good tool for those wanting to
> > either enforce or learn the functional style.
>
> > Branko

"My point today is that, if we wish to count lines of code, we should not regard them as "lines produced" but as "lines spent": the current conventional wisdom is so foolish as to book that count on the wrong side of the ledger" ~ Dijkstra

You'd have to exclude *any* pre-compiled library as being unsafe, unless you were somehow able to flag stuff as immutable within the classfile.

Unless I'm mistaken, you couldn't run anything then. ;-)Scala needs the Java runtime jars to begin with, they will never have such annotations. Further, 100% of the scala universe right now have no such annotations, and at any given time the libs you would be using would not have such annotations. Sounds like a brief exercise in frustration followed by abandonment.
Futher, this facility would be required at runtime since class files could be loaded dynamically.Clearly, this is all heading down the "effects system" path, which would be great, but I think that the horses have all escaped and closing the barn doors won't help. Lukas Rytz talked about this at Scala days (http://days2011.scala-lang.org/node/252) (are the videos available yet?), and I'm not bullish on the prospect for the success of this. I asked him specifically about run-time loaded classes and he said that would certainly subvert the type system (w.r.t. effects annotation). This isn't to say the an effects system for Scala should be abandoned, it's just hard to imagine it being, er..., effective ;-).
-- Jim Powers

You'd have to exclude *any* pre-compiled library as being unsafe, unless you were somehow able to flag stuff as immutable within the classfile.

Unless I'm mistaken, you couldn't run anything then. ;-)

That was indeed my point :)

Scala needs the Java runtime jars to begin with, they will never have such annotations. Further, 100% of the scala universe right now have no such annotations, and at any given time the libs you would be using would not have such annotations. Sounds like a brief exercise in frustration followed by abandonment.
Futher, this facility would be required at runtime since class files could be loaded dynamically.Clearly, this is all heading down the "effects system" path, which would be great, but I think that the horses have all escaped and closing the barn doors won't help. Lukas Rytz talked about this at Scala days (http://days2011.scala-lang.org/node/252) (are the videos available yet?), and I'm not bullish on the prospect for the success of this. I asked him specifically about run-time loaded classes and he said that would certainly subvert the type system (w.r.t. effects annotation). This isn't to say the an effects system for Scala should be abandoned, it's just hard to imagine it being, er..., effective ;-).

So I can see something like this being automatically added by the compiler:- A `@pure` annotation, on methods (and functions) without side-effects (only calling other @pure methods and using Immutable types)
- An `Immutable` marker trait, for AnyVals and classes/traits that only contain @pure methods and non-var references to other Immutable typesPerhaps controversially, both of these should also be manually addable; to support interop with Java classes where the compiler wouldn't be able to make such an analysis (and old Scala code that predates the system).
If you flag something as @pure or Immutable when it isn't, then the compiler should emit an error if it's able to clearly demonstrate that you're wrong (so this can be used as a way to sanity-check your assumptions). If you lied, but the compiler can't deduce that, then any resulting problems are entirely self-inflicted.

This would then allow us to enforce such constraints as [K <: Immutable] for the key of a HashMap, which would be nice.

If you flag something as @pure or Immutable when it isn't, then the compiler should emit an error if it's able to clearly demonstrate that you're wrong (so this can be used as a way to sanity-check your assumptions). If you lied, but the compiler can't deduce that, then any resulting problems are entirely self-inflicted.

If I had to hazard a guess "lying" would be the default, and in many cases, the lie will be (practically) irrelevant. At the very least logging, then followed by the potential (directly or indirectly) to throw exceptions, the list goes on from there. Then there continues to be the not so small matter of dynamic class loading for which none of this machinery works. :-(.

This would then allow us to enforce such constraints as [K <: Immutable] for the key of a HashMap, which would be nice.

It is very much unclear to me how much faith I could place in the compiler's (alone) claim of immutability of anything, given the ready access to methods, techniques, and pathways to subvert such claims.

>> You'd have to exclude *any* pre-compiled library as being unsafe, unless
>> you were somehow able to flag stuff as immutable within the classfile.
>
> Unless I'm mistaken, you couldn't run anything then. ;-)

> Scala needs the Java runtime jars to begin with, they will never have such
> annotations. Further, 100% of the scala universe right now have no such
> annotations, and at any given time the libs you would be using would not
> have such annotations. Sounds like a brief exercise in frustration followed
> by abandonment.

But that only matters for *fields* of your class. It matters little if
a method contains a call to java.util.Date -- mutable and non-thread
safe -- for instance, as long as the fields are all immutable, the
object will remain immutable.

Stuff like business logic could easily benefit from it if Scala's
standard library had the proper annotations.

> Futher, this facility would be required at runtime since class files could
> be loaded dynamically.

Anything decided only at run-time would be automatically excluded from
the "immutable" flag. But, more importantly, that's almost beside the
point -- you can mutate private immutable fields with reflection,
which would just as surely defeat the immutable flag -- if your
intention is *enforcement*, instead of *error detection*. This is the
equivalent of calling asInstanceOf in Scala -- at that point, you
threw type safety out of the window.

> Scala needs the Java runtime jars to begin with, they will never have such
> annotations. Further, 100% of the scala universe right now have no such
> annotations, and at any given time the libs you would be using would not
> have such annotations. Sounds like a brief exercise in frustration followed
> by abandonment.

But that only matters for *fields* of your class. It matters little if
a method contains a call to java.util.Date -- mutable and non-thread
safe -- for instance, as long as the fields are all immutable, the
object will remain immutable.

Unless I'm misinterpreting what you are saying here, I believe this particular issue has come up on the list before and this claim is not true:
scala> :paste// Entering paste mode (ctrl-D to finish)
class FauxImmutable { val a = Array(1,2,3)
}def pia(a:Array[Int]) {
println(a.mkString("Array(",",",")"))}
val test = new FauxImmutablepia(test.a)
test.a(0) = 0test.a(1) = 0
test.a(2) = 0pia(test.a)
// Exiting paste mode, now interpreting.Array(1,2,3)Array(0,0,0)
Any object in a val field that allows mutation will continue to do so.

On Wed, Jun 29, 2011 at 17:23, Jim Powers wrote:
>>
>> But that only matters for *fields* of your class. It matters little if
>> a method contains a call to java.util.Date -- mutable and non-thread
>> safe -- for instance, as long as the fields are all immutable, the
>> object will remain immutable.
>
> Unless I'm misinterpreting what you are saying here, I believe this
> particular issue has come up on the list before and this claim is not true:
> scala> :paste
> // Entering paste mode (ctrl-D to finish)
> class FauxImmutable {
> val a = Array(1,2,3)

On 29/06/11 22:56, Branks wrote:
> Hi,
>
> Just wondering whether or not it would be good to have an
> "immutable" compiler option added to scalac. For example:
>
> scalac -Ximmutable
>
> Any scala source that makes use of vars or immutable types would
> fail to compile. This could also be a good tool for those wanting
> to either enforce or learn the functional style.
>
> Branko
What about when I call java.io.InputStream#next?

OK, cool, agreement there. Now we're back to the issue of the original post and one need only establish the, not completely easily done, claim that some value is immutable. Here's where my confidence drops.
-- Jim Powers

On 30/06/11 00:50, Jim Powers wrote:
> Interesting, but how far would the consequences go?
>
> - Java-based stuff (ArrayList, Java HashMap, StringBuilder, ...
> )? - Arrays? Somewhat pervasive when
> working with Java libraries. - Builders? a number of .toSeq
> .toStream (etc...) things use builders internally, so the rule
> would be that the libraries could "cheat" but you couldn't?
These libraries could use the equivalent of ST. That is they use
in-memory variables, but do not use I/O.
>
> Just curious.
>
Of course, this proposal overlooks the fact that variables are
pervasive. That is, as soon as you use one, then anything that uses
that varies (modulo ST).

If you're looking for an effect-tracking system, which is what this
is, take a look at scalaz.effects. It includes both IO and ST type
constructors to *fail the compiler* if you use variables in a way that
is incorrect (assuming the language subset without var keyword).

These libraries could use the equivalent of ST. That is they use
in-memory variables, but do not use I/O.
>
> Just curious.
>
Of course, this proposal overlooks the fact that variables are
pervasive. That is, as soon as you use one, then anything that uses
that varies (modulo ST).

If you're looking for an effect-tracking system, which is what this
is, take a look at scalaz.effects. It includes both IO and ST type
constructors to *fail the compiler* if you use variables in a way that
is incorrect (assuming the language subset without var keyword).

I have seen it and read Runar's write-up about it (then hit the paper trail), so I'm convinced, but the vast ocean that is code on the JVM does not play by the ST (or IO) monad rules. The intent of the original post is to enable a compile time flag that asks the compiler to assert some kind of immutability guarantee checks that one could trust. Given the light-weight, voluntary effects-system that the Scala compiler guys are looking at I would have some confidence that if the compiler spat out that a requirement for a pure function could not be met that such a claim would ultimately be shown to be true. It's the substantial possibility of false "pass checks" cases that doesn't leave me with a good feeling about this idea. It feels like it's kinda too late for this. For your "own" (or some relatively controlled code base) code, sure, and the Scalaz approach seems to fit the bill better than pretty much all the options I've seen.
Just sayin'-- Jim Powers

One way to deal with "impure vars" would be to convert them into "pure vals". See my paper on purely functional structured programming (http://arxiv.org/abs/1007.3023).- Steven On 29.06.2011, at 23:37, Tony Morris wrote:

-----BEGIN PGP SIGNED MESSAGE-----Hash: SHA1

On 30/06/11 00:50, Jim Powers wrote:

Interesting, but how far would the consequences go?

- Java-based stuff (ArrayList, Java HashMap, StringBuilder, ...

<all other things not scala>)? - Arrays? Somewhat pervasive when

working with Java libraries. - Builders? a number of .toSeq

.toStream (etc...) things use builders internally, so the rule

would be that the libraries could "cheat" but you couldn't?

These libraries could use the equivalent of ST. That is they usein-memory variables, but do not use I/O.

Just curious.

Of course, this proposal overlooks the fact that variables arepervasive. That is, as soon as you use one, then anything that usesthat varies (modulo ST).

If you're looking for an effect-tracking system, which is what thisis, take a look at scalaz.effects. It includes both IO and ST typeconstructors to *fail the compiler* if you use variables in a way thatis incorrect (assuming the language subset without var keyword).

On 30/06/11 08:02, Jim Powers wrote:
> On Wed, Jun 29, 2011 at 5:37 PM, Tony Morris
> wrote:
>
>> These libraries could use the equivalent of ST. That is they use
>> in-memory variables, but do not use I/O.
>>>
>>> Just curious.
>>>
>> Of course, this proposal overlooks the fact that variables are
>> pervasive. That is, as soon as you use one, then anything that
>> uses that varies (modulo ST).
>>
>> If you're looking for an effect-tracking system, which is what
>> this is, take a look at scalaz.effects. It includes both IO and
>> ST type constructors to *fail the compiler* if you use variables
>> in a way that is incorrect (assuming the language subset without
>> var keyword).
>
>
> I have seen it and read Runar's write-up about it (then hit the
> paper trail), so I'm convinced, but the vast ocean that is code on
> the JVM does not play by the ST (or IO) monad rules.

You mean they tell lies in their type right? Yeah, you need to wrap
them. The given proposal is not going to help this -- it will make
things worse in fact.

> The intent of the original post is to enable a compile time flag
> that asks the compiler to assert some kind of immutability
> guarantee checks that one could trust.

It cannot be trusted. It is not just "var" that circumvents this
system. Remember, variables are *pervasive* -- this is why we expend
so much effort trying to track them.

If I wanted to circumvent this system, I'd simply download this
third-party library (which compiled with warning):

class Var[A] {
var a: A = _
}

Now I can use var to my heart's content.

So, I will point out with a bit more rigour -- the scalaz
effect-tracking may not be as trustable as we like (var still exists),
but it is the best available to date, even if this proposal were to be
implemented.

> Given the light-weight, voluntary effects-system that the Scala
> compiler guys are looking at I would have some confidence that if
> the compiler spat out that a requirement for a pure function could
> not be met that such a claim would ultimately be shown to be true.
> It's the substantial possibility of false "pass checks" cases that
> doesn't leave me with a good feeling about this idea. It feels
> like it's kinda too late for this. For your "own" (or some
> relatively controlled code base) code, sure, and the Scalaz
> approach seems to fit the bill better than pretty much all the
> options I've seen.

It also works with existing Java code. I am using it in production
right now in fact, with JDBC.

This looks like ST. Is it different?
Lazy Functional State Threads -- Launchbury, Peyton-Jones

On 30/06/11 08:08, Steven Obua wrote:
> One way to deal with "impure
vars" would be to convert them into
> "pure vals". See my paper on purely functional structured
> programming (http://arxiv.org/abs/1007.3023).
>
> - Steven
>
> On 29.06.2011, at 23:37, Tony Morris wrote:
>
>>
> On 30/06/11 00:50, Jim Powers wrote:
>>>> Interesting, but how far would the consequences
go?
>>>>
>>>> - Java-based stuff (ArrayList, Java HashMap,
StringBuilder,
>>>> ... <all other things not scala>)? -
Arrays? Somewhat
>>>> pervasive when working with Java libraries. -
Builders? a
>>>> number of .toSeq .toStream (etc...) things use
builders
>>>> internally, so the rule would be that the
libraries could
>>>> "cheat" but you couldn't?
> These libraries could use the equivalent of ST. That is
they use
> in-memory variables, but do not use I/O.
>>>>
>>>> Just curious.
>>>>
> Of course, this proposal overlooks the fact that variables
are
> pervasive. That is, as soon as you use one, then anything
that
> uses that varies (modulo ST).
>
> If you're looking for an effect-tracking system, which is
what
> this is, take a look at scalaz.effects. It includes both IO
and ST
> type constructors to *fail the compiler* if you use
variables in a
> way that is incorrect (assuming the language subset without
var
> keyword).
>
>>

Yes, it is actually very different; ST is about doing computations that involve state. My paper is about a concise syntax for computations that actually do NOT involve state but are often mistaken to be stateful computations just because they contain "var"s. For example, take the calculation of the gcd:

On 30/06/11 18:22, Branko Juric wrote:
> After reading all these comments it seems that it is not possible to provide only positive assurance guarantees on immutability at the compiler level.
It is. It just requires significant changes to the existing language.

On 30/06/11 19:13, Branko Juric wrote:
> Not possible now but coming soon?
I don't imagine there is any plan to change the language to accommodate
a usable/practical effect-tracking system. You can make do with assuming
the language subset absent the var keyword and scalaz.effects.{IO, ST}
and at least get a lot further than you do in other type systems.