Interoperation with Java

Ceylon is designed to execute in a virtual machine environment
(loosely speaking, in an environment with built-in garbage
collection), but it doesn't have its own native virtual machine.
Instead, it "borrows" a virtual machine designed for some other
language. For now the options are:

the Java Virtual Machine, or

a JavaScript virtual machine.

In this chapter, we're going to learn about how to interoperate
with native code running on the Java Virtual Machine, and focus
on all the nitty-gritty details and corner cases involved in
inter-language interop. In the next chapter we'll
discuss JavaScript.

Defining a native Java module

Many Ceylon modules that call native Java code are designed
to execute only on the JVM. In this case, we declare the whole
Ceylon module as a native JVM module using the native
annotation.

It's not necessary to annotate individual declarations in a
module that is already declared as a native JVM module.

Importing Java modules containing native code

A Ceylon program can only use native code belonging to a module.
Ceylon doesn't have the notion of a global class path, and not
every Ceylon program can be assumed to be executing on the JVM,
so dependencies to native Java code must always be expressed
explicitly in the module descriptor.

The native Java modules your Ceylon program depends on might
come from a Ceylon module repository, from a Maven repository,
or they might be part of the JDK or Android SDK.

Depending on the JDK

The Java SE Development Kit (JDK) is represented in Ceylon as
a set of modules, according to the modularization proposed by
the Jigsaw project. So to make use of the JDK we need to
import one or more of these modules. For example:

Depending on the Android SDK

For Android development, it's necessary to use the
Ceylon Android plugin for Gradle to create
a Ceylon module repository containing the bits of the Android
SDK that are needed to compile an app written in Ceylon.
This repository will be created in the subdirectory
build/intermediates/ceylon-android/repository of your
Ceylon Android project. This repository must be specified as
the source of all Android-related Java modules using --rep,
and as the provider of the Java SDK using the command-line
argument --jdk-provider of ceylon compile.

However, when compiling using Gradle, or from inside Android
Studio, the repository is created by the build, and the
compiler option is set automatically, so you don't need to
mess with it explicitly. You can find it in .ceylon/config,
if you're interested:

Note: Android itself has an immensely complicated toolchain,
and Android apps cannot reasonably be compiled using only
commmand line tools. In practice, you'll always use Gradle to
build an Android app.

A basic boilerplate module descriptor for Android development
looks like this:

Depending on Java EE

The Java EE APIs are available in Ceylon Herd, so you can import
them like this:

import javax.javaeeapi "7.0";

Alternatively, you can get them from Maven:

import maven:javax:"javaee-api" "7.0";

The second approach might work better if you're also importing
other related Java modules from Maven.

Interoperation with Java types

Calling and extending Java types from Ceylon is mostly
completely transparent. You don't need to do anything "special"
to use or extend a Java class or interface in Ceylon, or to
call its methods.

There's a handful of things to be aware of when writing Ceylon
code that calls a Java class or interface, arising out of the
differences between the type systems of the two languages.

Certain very abstract Java supertypes are mapped to Ceylon types

There are three Java classes and one interface which simply
can't be used at all in Ceylon code, because Ceylon provides
types which are exactly equivalent in the package
ceylon.language. When these types occur in the signature
of an operation in Java, they're always represented by the
equivalent Ceylon type:

Almost all of the time, this behavior is completely intuitive.
But there's two wrinkles to be aware of...

Gotcha!

According to these rules, all conversions from a Java primitive
to a Ceylon type are widening conversions, and are guaranteed
to succeed at runtime. However, conversion from a Ceylon type
to a Java primitive type might involve an implicit narrowing
conversion. For example, if:

a Ceylon Integer is assigned to a Java int or short,

a Ceylon Float is assigned to a Java float, or if

a Ceylon UTF-32 Character is assigned to a Java 16-bit
char,

the assignment can result in silent overflow or loss of
precision at runtime.

Note: it is not a goal of Ceylon's type system to warn about
operations which might result in numeric overflow. In general,
almost any operation on a numeric type, including + or *,
can result in numeric overflow.

Gotcha again!

There's no mapping between Java's wrapper classes like
java.lang.Integer or java.lang.Boolean and Ceylon basic
types, so these conversions must be performed explicitly by
calling, for example, longValue() or booleanValue(), or
by explicitly instantiating the wrapper class, just like you
would do in Java when converting between a Java primitive
type and its wrapper class.

This is mainly important when working with Java collection
types. For example, a Java List<Integer> doesn't contain
CeylonIntegers.

Tip: converting Java strings

Explicitly converting between String and Java's
java.lang.String is easy:

the .string attribute of a Java string returns a Ceylon
string, and

one of the constructors of java.lang.String accepts a
Ceylon String, or, alternatively,

the function Types.nativeString() in the package java.lang
converts a Ceylon string to a Java string without
requiring an object instantiation.

Tip: converting Java primitive wrapper types

Likewise, conversions between Ceylon types and Java primitive
wrapper types are just as trivial, for example:

the .longValue() methods of java.lang.Long and
java.lang.Integer return a Ceylon Integer, and

the constructors of java.lang.Integer and java.lang.Long
accept a Ceylon Integer, as do the static valueOf()
methods of these types.

Likewise:

the .doubleValue() methods of java.lang.Double and
java.lang.Float return a Ceylon Float, and

the constructors of java.lang.Double and java.lang.Float
accept a Ceylon Float, as do the static valueOf()
methods of these types.

The story is similar for other primitive wrapper types.

Tip: using the small annotation

If, for some strange reason, you really need a 32-bit int
or float at the bytecode level, instead of the 64-bit long
or double that the Ceylon compiler uses by default, you can
use the small annotation.

small Integer int = string.hash; //a 32-bit int

You can also use small to represent a Character as a
16-bit char at the bytecode level, instead of as a 32-bit
int.

small Character char = charArray.get(0); //a 16-bit char

It's important to understand that small Integer isn't a
different type to Integer. So any Integer is directly
assignable to a declaration of type small Integer, and
the compiler will silently produce a narrowing conversion,
which could result in silent overflow or loss of precision
at runtime.

Note also that small is defined as a hint, and may be
completely ignored by the compiler. (And, indeed, it is
always ignored when compiling to JavaScript.)

Java array types are represented by special Ceylon classes

Since there are no primitively-defined array types in Ceylon,
arrays are represented by special classes. These classes are
considered to belong to the package java.lang. (Which belongs
to the module java.base.) So these classes must be explicitly
imported!

boolean[] is represented by the class BooleanArray,

char[] is represented by the class CharArray,

long[] is represented by the class LongArray,

int[] is represented by the class IntArray,

short[] is represented by the class ShortArray,

byte[] is represented by the class ByteArray,

double[] is represented by the class DoubleArray,

float[] is represented by the class FloatArray, and,
finally,

T[] for any object type T is represented by the class
ObjectArray<T>.

We can obtain a Ceylon Array without losing the identity
of the underlying Java array.

You can think of the ByteArray as the actual underlying
byte[] instance, and the Array<Byte> as an instance of
the Ceylon class Array that wraps the byte[] instance.

The module ceylon.interop.java contains a raft of
additional methods for working with these Java array types.

Java object types and null values

Java's primitive types do not hold null values, so any
primitive type is mapped to a non-null Ceylon type. The
situation is a bit more complicated for Java object and
array types, which are treated differently depending upon
whether they occur:

as a method return type or field type, or

as a method parameter type.

Non-primitive parameter types are treated as nullable

Java method signatures offer no information about whether a
method parameter accepts a null value, except in the very
special case of a parameter with primitive type. Therefore,
Ceylon assigns an optional type to every parameter of
non-primitive type.

Null return values are checked at runtime

Likewise, Java field or method types offer no information about
whether a field or method can produce a null value, except,
again, in the special case of a primitive type. But it would be
unacceptably intrusive for Ceylon to treat almost every Java
method or field as returning an optional type, forcing you to
explicitly check for null every time you call Java! So Ceylon
doesn't do that. Instead, it treats Java methods and fields as
having non-optional type.

But that's, of course, unsound from the point of view of the
type system of Ceylon. A method can still return null at
runtime!

Therefore, the compiler must insert runtime null value checks
wherever Ceylon code calls a Java function that returns an
object type, or evaluates a Java field of object type, and
assigns the result to an non-optional Ceylon type.

In this example, no runtime null value check is performed,
since the return value of System.getProperty() is assigned to
an optional type:

These runtime checks ensure that null can never unsoundly
propagate from native Java code with unchecked null values
into Ceylon code with checked null values, resulting in an
eventual NullPointerException in Ceylon code far from the
original call to Java.

Java types annotated @Nullable are exposed as optional types

There are now a number of Java frameworks that provide
annotations (@Nullable and @Nonnull/@NonNull/@NotNull)
for indicating whether a Java type can contain null values.
The Ceylon compiler understands these annotations.

So, as an exception to the above discussion, when one of
these annotations is present, Java null values are checked
at compile time, and no runtime checks are necessary.

Java methods, properties, and constants

Ceylon makes use of the JavaBeans conventions when representing
a Java class.

Java properties are exposed as Ceylon attributes

A getter or getter/setter pair belonging to a Java class will
appear to a Ceylon program as an
attribute.
For example:

to call the single-argument setter, use an assignment
statement like foo.bar = bar, but

to call the two-argument setter, use a method call like
foo.setBar(bar,baz).

Other Java methods are exposed as Ceylon methods

An Java method that doesn't follow the JavaBeans property
conventions is represented to the Ceylon program as a regular
method.

Gotcha: calling Java methods from Ceylon

There's two minor limitations to be aware of when calling Java
methods from Ceylon.

You can't call Java methods using the named argument syntax,
since Java 7 doesn't expose the names of parameters at
runtime (except for code compiled on Java 8 with a special
compiler switch explicitly enabled).

You can't obtain a method reference, nor a static method
reference, to an overloaded method.

Methods accepting a SAM interface

Java has no true function types, so there's no equivalent to
Ceylon's Callable interface in Java. Instead, Java features
SAM (Single Abstract Method) conversion where an anonymous
function is converted by the compiler to an instance of an
interface type like java.util.function.Predicate which
declares only one abstract method.

Thus, there are many operations in the Java SDK and other
Java libraries which accept a SAM interface. For example,
Stream.filter() accepts a Predicate. Such methods are
represented in Ceylon as an overloaded method:

one overload accepts the SAM, like
Stream<T> filter(Predicate<in T> predicate), and

the second overload accepts a Ceylon function type, like
Stream<T> filter(Boolean(T) predicate).

Thus, we can pass anonymous functions and function references
to such Java APIs without needing to explicitly implement the
SAM interface type.

As you can see, use of the Java streams API is completely
natural in Ceylon.

Tip: getting a function from a SAM

There's no inverse conversion from a SAM type to a Ceylon
function type. A Predicate<T> can't be passed directly to
an operation that expects a Boolean(T). But this is
perfectly fine, because it's completely trivial to perform
the conversion explicitly, just by taking a reference to the
test method of Predicate:

Java constants and enum values

Java constants like Integer.MAX_VALUE and Java enum values,
like RetentionPolicy.RUNTIME, follow an all-uppercase naming
convention. Since this looks rather alien in Ceylon code, it's
acceptable to refer to them using camel case. This:

Tip: switching over Java enum types

A Java enum type is treated as as enumerated class. Each
enumerated value of the enum type is represented as a
value constructor
of the class. Thus, it's possible to switch over the members
of a Java enum, just like you can in Java.

Java generic types

A generic instantiation of a Java type like java.util.List<E>
is representable without any special mapping rules. The Java
type List<String> may be written as List<String> in Ceylon,
after importing List and String from the module java.base.

Wildcards and raw types are represented using use-site variance

In pure Ceylon code, we almost always use
declaration-site variance.
However, this approach doesn't work when we interoperate with
Java generic types, which are by nature all invariant. In Java,
covariance or contravariance is represented at the point of use
of the generic type, using wildcards. Therefore, Ceylon also
supports use-site variance (wildcards).

Use-site variance is indicated using the keywords out and in:

List<out Object> has a covariant wildcard, and is
equivalent to List<? extends Object> in Java, and

Topic<in Object> has a contravariant wildcard, and is
equivalent to Topic<? super Object> in Java.

Java raw types are also represented with a covariant wildcard.
The raw type List is represented as List<out Object> in
Ceylon.

Note that for any invariant generic type Type<X>, the
instantiations Type<out Anything> and Type<in Nothing> are
exactly equivalent, and are supertypes of every other
instantiation of Type.

Wildcard types are unavoidable when interoperating with Java,
and perhaps occasionally useful in pure Ceylon code. But we
recommend avoiding them, except where there's a really good
reason.

Gotcha!

Instances of Java generic types don't carry information about
generic type arguments at runtime. So certain operations which
are legal in Ceylon are rejected by the Ceylon compiler when
a Java generic type is involved. For example, the following code
is illegal, since JList is a Java generic type, with erased
type arguments:

However, a Ceylon class which implements the Java types Foo
and Baris not legal,
since a Ceylon class may not define an operation that refines
two completely different but same-named operations unless they
both descend from a common supertype definition:

class Baz() satisfies Foo & Bar {
//error: May not inherit two declarations with the
//same name that do not share a common supertype
test() => true;
}

In this case, it's necessary to either:

split Baz into two classes,

define an intermediate Java interface that inherits Foo
and Bar, or

define a common superinterface of Foo and Bar that
declares test().

Java annotations

Java annotations may be applied to Ceylon program elements.
An initial lowercase identifier must be used to refer to the
Java annotation type.

Tip: specifying arguments to anotation parameters of type Class

Annotation values of type java.util.Class may be specified
by passing the corresponding Ceylon ClassDeclaration. For
example, you would use type = `class Person` where you
would have used type = Person.class in Java.

Java language modifiers are represented as annotations

The following Java language modifiers do not naturally
correspond to any annotation in ceylon.language:

Therefore, if you want to use one of these JVM-specific
modifiers in Ceylon, you must explicitly import it:

import java.lang { volatile }
volatile variable value counter = 0;

Of these annotations, transient and volatile are the most
commonly-used in Ceylon code. We discourage direct use of
synchronized, which is extremely vulnerable to deadlocks,
preferring the use of java.util.concurrent.

Syntax sugar that applies to Java types

Certain syntactic constructs that are defined by the language
specification in terms of types defined in ceylon.language
are also supported for similar Java types. These constructs
are:

the for loop and comprehensions,

resource expressions in try,

the element lookup and in operators, and

the "spread" operators *. and *.

Java Iterable or array in for

It's possible to use a java.lang.Iterable or Java array in
a Ceylon for statement or comprehension.

Imagine how great it would be to be able to write a
comprehension involving a Java collection in Java itself!

Wait, a quick reminder...

Gotcha! (redux)

As we already noted above, for does nothing special to the
type of the iterated elements of a Java array or Iterable:
if the elements are Java Strings, they're not magically
transformed to Ceylon Strings:

However, copying collections by nature involves memory
allocation and this can be slow. A more efficient approach is
to wrap the Java collection. Fortunately, Ceylon has a library
for that.

Utility functions and classes

The class Types, which is considered to belong to the package
java.lang in the module java.base offers some extremely useful
static utility methods. For example, Types.nativeString()
converts Ceylon strings to java.lang.Strings and Types.charArray()
converts Ceylon strings into Java char[] arrays.

In the module ceylon.interop.java you'll find a suite
of useful utility functions and classes for Java interoperation.
For example, there are classes that adapt between Ceylon
collection types and Java collection types.

Tip: getting a java.util.Class

An especially useful function is Types.classForType, which
obtains an instance of java.util.Class for a given type.

Similarly there are CeylonStringIterable, CeylonIntegerIterable,
CeylonFloatIterable,CeylonByteIterable and CeylonBooleanIterable
classes which as well as converting the iterable type also
convert the elements from their Java types to the corresponding
Ceylon type.

Additional support for Java interoperation

Ceylon provides the following additional features to support
interoperation with the Java platform.

META-INF and WEB-INF

Some Java frameworks and environments require metadata packaged
in the META-INF or WEB-INF directory of the module archive,
or sometimes even in the root directory of the module archive.
We've already seen how to package resources in Ceylon module
archives by placing the resource files in the module-specific
subdirectory of the resource directory, named resource by
default.

Then, given a module named net.example.foo:

resources to be packaged in the root directory of the module
archive should be placed in resource/net/example/foo/ROOT/,

resources to be packaged in the META-INF directory of the
module archive should be placed in
resource/net/example/foo/ROOT/META-INF/, and

resources to be packaged in the WEB-INF directory of the
module archive should be placed in
resource/net/example/foo/ROOT/WEB-INF/.

Interoperation with Java serialization

When compiling a Ceylon class, the compiler adds the supertype
java.io.Serializable to the generated Java class, along with
a package-private default constructor. Neither the supertype,
nor the constructor, are visible to other Ceylon code. But
they're enough to make the Ceylon class serializable via
Java's built-in binary serialization APIs.

Of course, if your Ceylon object holds a reference to some
other object that's not serializable, you still won't be able
to serialize the Ceylon object!

When using this sort of framework in Ceylon, you'll probably
need to enable a special compiler mode which
slightly changes the way the compiled Ceylon class represents
its internal state using Java fields.

With EE mode activated, Ceylon classes should work
transparently with these frameworks and libraries. (Note:
we've already tested all the above listed technologies with
Ceylon, to be sure they work correctly!)

In many cases EE mode is activated implicitly when you
import a Java module or use a Java EE annotation.

Alternative module runtimes

When you execute a Ceylon program using ceylon run, it
executes on a lightweight module container that is based on
JBoss Modules. But it's also possible to execute a Ceylon
module in certain other module containers, or without any
module container at all.

Tip: running your program without a Ceylon installation

Don't forget that you can run
a Ceylon program assembled using ceylon assemble --include-runtime
on a machine with no Ceylon installation. In this case, the
JBoss Modules-based module container is included in the .cas
archive itself.

Deploying Ceylon on Jigsaw

The command line tool ceylon jigsaw deploys a module and
all its dependencies to a Jigsaw-style mlib/ directory. Since
Jigsaw is not yet released, and is still undergoing changes, this
functionality is considered experimental.

Deploying Ceylon on Java EE or WildFly Swarm

These command line tools make it easy to assemble a Ceylon module
that makes use of Java EE APis in a Java EE environment:

ceylon war repackages a module and its dependencies as a
Java EE .war web archive, and

Tip: including static web content

To include a directory containing static web resources like HTML,
image, and JavaScript files in an archive generated by ceylon war
or ceylon swarm, use --resource-root or -R, for example:

ceylon war -R=web com.acme.todolist

Deploying Ceylon as a fat jar

To run a Ceylon program on a machine with no Ceylon distribution
available, and without the runtime module isolation provided by
JBoss Modules, the command line tool ceylon fat-jar is
indispensable. The command simply assembles a Java .jar archive
that contains a Ceylon module and everything it depends on at
runtime.

Publishing Ceylon via Maven

Finally, the ceylon maven-export command assembles a Maven
repository containing a Ceylon module and its dependencies. This is
most useful if we want to make a Ceylon module available to a Java
project that is built using Maven.

However, this isn't the only way to integrate Ceylon modules into a
Maven build.

Ceylon and Maven

Every compiled Ceylon module archive includes generated Maven
metadata, in META-INF/maven/groupId/artifactId/pom.xml and
META-INF/maven/groupId/artifactId/pom.properties.

You can specify the Maven group id and artifact id in your module
descriptor:

If you build your Ceylon modules using Maven, or if you use
ceylon maven-export, you can easily publish the resulting module
archives to a public Maven repository such as Maven Central.

All Ceylon SDK platform modules are already available in Maven Central,
under the group id org.ceylon-lang. (They're also available in
under the same group id
in the Herd Maven repository.) For example, ceylon.collection has the
group id org.ceylon-lang and artifact id ceylon.collection.