Although there is room for performance tuning,
the prototype supports the full
Closures (v0.5) specification. Based on your
feedback, there are some changes in the prototype
suitable for a future update of the specification:

Renamed Unreachable to Nothing

We adopt the name used by Scala to represent the same concept.

Removed support for the type null

We used null as a placeholder for an exception type when
none can be thrown. The type Nothing now serves that
purpose; null is no longer supported as the name of a
type.

Overhauled restricted versus unrestricted

In the specification, an interface is considered restricted
if it extends a marker interface. Unfortunately, the specification
only provides a syntax for function type interfaces that are
unrestricted. We modified the syntax so that a function type written
using the => token designates a restricted function type,
while one written using the newly introduced ==> token
represents an unrestricted function type. This allows programmers to
easily write APIs that restrict (or don't restrict) the operations of
closure expressions passed as parameters.

Refined restrictions

We modified the distinction between restricted and unrestricted
closures. As before, it is not legal to convert an unrestricted
closure to a restricted interface type, nor is it legal to
break, continue, or return from
inside a restricted closure to a target outside the closure. However,
a restricted closure is allowed to refer to a non-final local variable
from an enclosing scope. In this case a warning is given unless one
of the following conditions holds:

The variable is not the target of any assignment, or

The variable is annotated @Shared

It is possible to suppress the warning by annotating some enclosing
construct @SuppressWarnings("shared").

Relaxed the closure conversion

In response to user feedback, we've relaxed the relationship between a
closure parameter's type and the target interface's parameter type.
Rather than requiring them to be of the same type, they are now
allowed to be related by an assignment conversion, including boxing or
unboxing.

for-qualified method declarations

The for keyword on a method declaration, meant to
introduce a control abstraction method that works like a loop, is now
treated syntactically like a modifier rather than appearing
immediately before the method name. This helps make the declaration
site more similar to the use site.

Added support for method references

We added extensive support for treating a reference to a method as
a closure using a newly introduced token #. The syntax
is borrowed from the FCM
proposal. The semantics are as follows:

A method reference written as

Primary # Identifier ( TypeList )

where the Primary designates an expression (as opposed to a type) is
treated the same as a closure

{ Type x0, Type x1 ... => tmp.Identifier(x0, x1 ...) }

or

{ Type x0, Type x1 ... => tmp.Identifier(x0, x1 ...); }

Where tmp is a temporary value that holds the computed value of the
primary expression. The former translation is used when the resolved
method has a non-void return type, while the latter is used when the
resolved method has a void return type.

If the primary resolves to a type, then this is translated to

{ Type x0, Type x1 ... => Primary.Identifier(x0, x1 ...) }

or

{ Type x0, Type x1 ... => Primary.Identifier(x0, x1 ...); }

when the resolved method is static, or

{ Primary x, Type x0, Type x1 ... => x.Identifier(x0, x1 ...) }

or

{ Primary x, Type x0, Type x1 ... => x.Identifier(x0, x1 ...); }

when the resolved method is an instance method.

In addition, optional explicit type arguments, between angle brackets,
may be placed immediately after the # token. These are
used directly in the translated method invocation to resolve the
method to be invoked.

Implemented a classfile format for the for qualifier

We've impleemnted a class file representation of the for
qualifier to support separate compilation.

Monday, March 17, 2008

Closures: Control Abstraction, Method References, Puzzler Solution

The Java Closures prototype now supports control abstraction and
implements restricted closures and function types. The syntax has
changed slightly. Also, as hinted in the
draft JSR
proposal, there is now support for eta abstraction, which
is called method reference in
Stephen
Colebourne's FCM proposal. We haven't updated the
specification, so this will serve
as a brief tutorial on the changes until we do. I don't know if this
will be the syntax we will end up with, but it will do for now. Finally,
we look at solutions to the closure puzzler in my previous post.

Control Abstraction

The first thing you'll notice when using the new prototype is that
the compiler gives a warning when a closure uses a local variable from an enclosing scope:

make sure the variable is not the target of any assignment expression; or

put @SuppressWarnings("shared") on an enclosing method or class; or

use an unrestricted closure, by using the ==> token
instead of the => token (when possible).

The => token builds a restricted closure that
triggers this warning. Restricted closures also do not allow a
break or continue statement to a target outside
the closure, nor a return statement from the enclosing method.
You will rarely want to write an unrestricted closure; many (but not all) of
the things you need to do with an unrestricted closure can be expressed more
clearly with a control invocation statement instead.

You're not allowed to assign an unrestricted closure to a restricted
interface. A number of existing JDK interfaces, such as
java.lang.Runnable, have been modified to be restricted.

In the less common case that you're writing a method intended to be used as a control
API, you can write a function type with the (new) ==> token to designate
an unrestricted function (interface) type. Let's do that to write a method,
with, that will automatically close a stream for us. The idea is to be able
to replace this code

Method References

A natural companion to closures is a way to refer to an existing method instead
of writing a closure that accepts the same arguments and just invokes the method. This is
sometimes known as eta
abstraction or method
references. We expect closures in their final form to include support for this
convenient feature, which is why it is called out in the
draft JSR proposal. The
latest version of the prototype supports this, with a syntax based on javadoc conventions.
Here are a few examples:

Tuesday, February 05, 2008

People experience the world in different ways, using different
senses. Some people view the world primarily through sight. Do you
see what I mean? Some through sound. Do you hear me? Some
experience the world only after thought and reflection. Do you
know what I mean? Some point out "smells" in code as a way of
criticizing its design.
This puzzle explores the question of whether people experiencing
the same thing through different senses are really experiencing
the same thing at all. What is the program's output, and why?

This puzzle uses Java
Closures, so if you want to try it you should download the
prototype here. You can
download the puzzle sources here. Comments
on this blog post will not be posted to avoid spoiling the puzzle.

Friday, January 11, 2008

We tend to think of programming languages in two categories:
"living languages" in which we should seriously consider
developing new code, and "legacy languages" that we mainly
use, if at all, because we have to maintain an existing code base. The
act of classifying a language into one or the other category helps us
decide what, if anything, we might consider doing to change the
language. If a language is primarily a legacy language, changes should
be aimed at making it easier to maintain and modify existing bodies of
code. A living language, on the other hand, also benefits from changes
that make it easier to design, develop, and maintain new code. Living
languages evolve to reduce accidental complexity.

"What does a high-level language accomplish? It frees a
program from much of its accidental complexity. An abstract program
consists of conceptual constructs: operations, datatypes, sequences,
and communication. The concrete machine program is concerned with bits,
registers, conditions, branches, channels, disks, and such. To the
extent that the high-level language embodies the constructs wanted in
the abstract program and avoids all lower ones, it eliminates a whole
level of complexity that was never inherent in the program at all."
-No Silver Bullet - Fred Brooks

Programs written in legacy languages tend to exhibit a high degree
of accidental complexity
[Code's
Worst Enemy, Steve Yegge][Mr. Yegge
meets Mr. Brooks, Brian C Cunningham].
Early in the life of a
language, the complexity of programs written in that language may
appear to be essential, but as we learn more about software
engineering and programming languages, we find patterns of complexity
appearing in the code that can be eliminated by improved languages.

A good example of this is garbage collection. In C and C++, memory
management is a pervasive concern. Smart pointers and destructors
help, but they do not significantly reduce the complexity of memory
management. In languages with garbage collection, most of the
complexity of memory management is assumed by the implementation of
the language. Most languages that have been introduced in the past ten
years support garbage collection.

What about Java? Is it a living language, or a
legacy language like Cobol? This question underlies much of the
debate about how to move the Java programming language forward, if at
all. Carl Quinn asked at the December 14, 2007, JavaPolis Future of
Computing Panel (to be published on
http://www.parleys.com): "How can we address the issue of
evolving the [Java] platform, language, and libraries without breaking
things?"

Neal Gafter:
"If you don't want to change the meaning of anything ever,
you have no choice but to not do anything. The trick is to minimize
the effect of the changes while enabling as much as possible. I think
there's still a lot of room for adding functionality without
breaking existing stuff..."

Josh Bloch: "My view of what really happens is a little bit
morbid. I think that languages and platforms age by getting larger and
clunkier until they fall over of their own weight and die very very
slowly, like over ... well, they're all still alive (though not
many are programming Cobol anymore). I think it's a great thing,
I really love it. I think it's marvelous. It's the cycle of
birth, and growth, and death. I remember James saying to me [...]
eight years ago 'It's really great when you get to hit the
reset button every once and a while.'"

Personally, I believe rumors of Java's demise are greatly
exaggerated. We should think of Java as a living language, and strive
to eliminate much of the accidental complexity of Java programs. I
believe it is worth adding support for closures and control
abstraction, to reduce such complexity of both the sequential and
concurrent aspects of our programs. At the same time, for completely
new code bases, we should also consider (and continue to develop)
newer languages such as Scala, which benefit from the lessons of Java.

About Me

Neal Gafter is a Computer Programming Language Designer, Amateur Scientist and Philosopher.
He works for Microsoft on the evolution of the .NET platform languages.
He also has been known to Kibbitz on the evolution of the Java language.
Neal was granted an OpenJDK Community Innovators' Challenge award for his design and
implementation of lambda expressions for Java.
He was previously a software engineer at Google working on Google Calendar, and a senior staff engineer at Sun Microsystems,
where he co-designed and implemented the Java language features in releases 1.4 through 5.0. Neal is coauthor of
Java Puzzlers: Traps, Pitfalls, and Corner Cases (Addison Wesley, 2005). He was a member of the C++ Standards
Committee and led the development of C and C++ compilers at Sun Microsystems, Microtec Research, and Texas Instruments.
He holds a Ph.D. in computer science from the University of Rochester.