Summary

Enhance the Java core library APIs using the new lambda language feature
to improve the usability and convenience of the library.

Goals

The primary goal is to modernize the general library APIs by adding
the use of Lambda in suitable locations. Most implementations will
be provided as extension methods upon existing classes. We will target
high-traffic areas of the library and add Lambda APIs where we think
it will have the most benefit. Ideally, Lambda would appear in the
API wherever a programmer familiar with Lambda would expect it to be.
Alternatively, we'd like mainstream programmers to stumble over Lambda
APIs in the library and think, "Oh cool, they added a Lambda here, and
that lets me solve my problem more easily."

A secondary goal is to inform the design of Lambda language feature by
using Lambda in library APIs, calling them from real code, evaluating
the results, and providing feedback to the Lambda language/compiler team.

Goals can be summarized as follows:

Introduce a new idiom, lambda functions, to the existing libraries;

Improve utility and convenience of libraries with lambda functions;

Demonstrate best practices for extension methods; and

Demonstrate innovation and evolution of the familiar core libraries.

Non-Goals

It is not a goal to use Lambda in every possible place it could be used,
nor will the scope extend beyond core libraries. For example, client,
XML, and CORBA are not covered by this effort.

Very little new functionality will be added to the core classes with this
enhancement -- only new ways of doing familiar tasks.

There are no specific goals for the proportion of APIs that are upgraded
to use Lambda, that is, nothing like "we will add Lambda to xx% of the
library."

Success Metrics

Success will be evaluated based upon the degree to which the new APIs and
features are adopted by developers. For complete success use of the Lambda
features will become the default idiom for using the core libraries by Java
developers.

Motivation

Java 8 will include a new language feature called Lambda. Having this feature
in the language is useful, but with just language changes for Lambda, the
platform is incomplete. It will make the platform much more valuable to have
Lambda support added to appropriate areas of the library API.

In the past several years, a variety of new programming languages have emerged
and are gaining popularity. Most of these languages have some kind of block,
closure, or first-class function construct. While Java is still the
#1 programming language, the common view is that it hasn't kept up with
recent developments in programming languages. This is well recognized as a
motivation for the Lambda language feature itself. However, it's also necessary
to consider library APIs that support the use of Lambda. The alternative
languages all have libraries, and their APIs are tuned to work well -- smoothly
and idiomatically -- with the closures or function objects provided by the
language.

Similarly, with Java, we expect that as time progresses the use of Lambda will
become widespread and various coding idioms will develop around this feature.
The Java library APIs will need to be enhanced to support idiomatic usage with
Lambda alongside their current, conventional usage.

Description

This project has two phases:

A survey phase to identify candidates for Lambda API enhancements; and

A scoping and implementation phase to prioritize and select a subset of the
candidates and to implement them.

Candidate Survey

There are several approaches to discovering candidate sites for adding
Lambda-based APIs. One approach is to examine other systems with similar
language features and to look at their libraries and see how they use closures
and functions.

Consider Ruby for instance. The Ruby language supports a variety of first
class function-like constructs. These are useful in and of themselves, in that
they enable programmers to use a functional style of programming, to create
higher order functions, to compose functions, etc. In addition, since the Ruby
class library was developed along with a language that supports first-class
functions, many APIs in the class library make use of them. We can look through
the Ruby class libraries to find examples that we can use as inspiration
for potential enhancements to the Java libraries. Some of these examples from
Ruby's class library include the following:

The Integer class has times, upto, downto, and step methods that
support a variety of arithmetic iteration constructs. This removes the
need to have a variety of such iteration constructs defined in the
language.

The File class has a variation of the open method that takes a
block. The file is opened, passed to the block, and is closed when the
block returns. This helps avoid resource leaks, and it reduces the need
to have special execute-around constructs in the language (such as Java
7's try-with-resources construct).

The HTTP API classes have several methods that take blocks. For example,
the HTTPResponse class has an each method that calls a block that takes
(header, value) pairs.

In the Tk binding for Ruby, the widget creation method new takes a block
that is executed in the context of the newly created widget, which
provides a convenient and concise way of initializing the widget.

It's clear that there are many opportunities for using Lambda outside the
collections classes. There is no requirement to implement Lambda-based APIs
everywhere that Ruby does, but given Ruby's popularity and mindshare it seems
reasonable to look to Ruby's class library for initial ideas. It might also be
reasonable to look examine other systems, such as Groovy, Scala, Python,
Clojure, and Smalltalk to find similar inspiration for Lambda use in library APIs.

Another approach is to look through existing Java code -- both within the
library and outside, e.g. from the Qualitas Corpus -- and do pattern matching
or synthesis to discover candidates. A set of techniques might be as follows:

Run through the APIs and add an each() or forEach() method to anything
that plausibly contains a collection of something, or that can be iterated
over, even if it's not a collection. For example, a forEachLine() method
might be added to java.nio.file.Files that calls a Lambda for each line in
that file.

Look for opportunities to use the execute-around idiom. For example, files
must be closed after use, locks must be unlocked after use, etc.

Look for classes that implement Iterable and consider whether a Lambda would
be useful to invert the flow of control.

Look for for-loops and while-loops in code and consider Lambda for inverting
the flow of control.

Identify abstract classes with one or two abstract methods, and consider
adding constructors or factories that takes lambdas. Also consider this
for interfaces. The pattern is where implementation is to be "filled in"
via overriding; consider how to convert this into passing a Lambda as a
parameter.

Scoping and Implementation

The resulting set of candidates will probably be too long to implement in the
JDK 8 time frame. They will need to be prioritized and then truncated to fit
into the available schedule, with a given set of resources (people) allocated
to the project. Prioritization can be according to subjective importance of the
library, but it would be good to have some usage data to back it up. For
example, it might be useful to do a survey over the Qualitas Corpus to
determine which areas of the library are the most heavily used.

Language Design Issues

During API development using Lambda, the following issues should be considered
and relevant information should be fed back to the Lambda language design team.

Exception transparency: do you find a need to write lambda expressions
that can throw checked exceptions? do you need extra sets of SAM types
that have 'throws' clauses, maybe generic 'throws E'? is the result
extremely tedious?

Variance: Utility methods for functional interfaces will probably make
heaver use of wildcards than anything else in the API; is this extremely
tedious? Would it be useful to have better/less verbose variance support?

Unboxed overriding: Would it be useful in the design of functional
interfaces to allow primitive/void return types to override reference
return types, such as Predicate<T> <: Function<T, Boolean> for example?

Abstract class functional interfaces: In the previous item about abstract
classes for, is it the case that there are so many useful candidates that
it would be nice to directly support abstract classes as functional
interfaces, rather than having to manually define a subclass?

Generic-method functional interfaces: Do you find it useful to have a
functional interface that has a generic method, like: interface MapFactory { <K,V> Map<K,V> make(); }. Note the significant point here
is that it is the method that is generic, not the interface. This is
not currently supported; the benefit is that new type arguments can be
inferred for every invocation, which may be useful in some applications.

Chaining inference: In expressions like 'foo().bar(23)', the type
arguments for 'foo' are inferred independently of the 'bar(23)' part of
the expression. We plan to improve the use of context in inference for
many cases, but we're still looking for some experience that would
justify taking it this far.

Method reference disambiguation: Do you find a need to explicitly
disambiguate method references (when the method is overloaded or there's
a static/instance clash), either by writing a full signature or giving up
and using a lambda expression instead?

Testing

Tests will need to be developed for each new API that is added to the system.
New APIs are largely independent of each other, so it should be simple to test
them individually. Furthermore, the API enhancements are likely to consist
largely of adding new methods that do not impact other APIs in the same class.
So, testing of these new APIs should be fairly straightforward and should not
impact existing tests that use current APIs.

Risks and Assumptions

Adding an additional programming idiom increases the complexity for new users.
If Lambda APIs are added piecemeal all over the system, they may tend to
diverge in style. It would probably be beneficial to do a survey first and to
develop a consistent style that is applied across the library.

One point that mitigates risk is that since the API enhancements are largely
independent of each other, it is possible to change scope without too much
project impact. That is, consider the scope to be a prioritized list of API
enhancements. A line will be drawn at a certain point depending upon project
schedule and staffing. If this line needs to move for any reason, this should
have no impact on work already done, and little replanning should be necessary
because the items are largely independent of each other. For the same reason,
it should also be possible to reprioritize the list without too much effort,
should new information become available while the project is underway.

Using these new APIs intermixed with the existing APIs has the prospect of
making user code bases more complicated to maintain.

Dependences

This work depends only on the actual lambda implementation.

Even this is a fairly soft dependency, as lambdas are expressed as SAMs in the
API. It's possible to add functional-interface-based APIs even before lambda
has been integrated, though it won't be possible for callers to use
lambda at that point.

Impact

JCP: These are all changes to the public API, so they will end up
modifying JCP-controlled specifications. These change to existing core
library classes should be small enough not to require their own JSR; they
should be covered under the platform JSR. The new classes required
(functional interfaces) will be designed and reviewed by the JSR 335
EG. Of course, the appropriate review processes must all be followed.

Documentation: There will probably need to be some new tutorials to show
how Lambda can be used with the new library constructs, or to have a
series of small examples illustrating common usage. It may be that
Lambdas are difficult to discern when looking through javadoc, so it
might be useful or necessary to have some new javadoc syntax or possibly
different javadoc output formatting to highlight in some unique way the
APIs that use lambda. An additional possibility is to make sure that the
javadoc for each of the new Lambda-accepting APIs includes an example of
its use.