Introduction

However, the current analysis is restricted to flow analysis within one method.
No assumptions can be made about

arguments flowing into a method

return values from method calls and

field reads.

In order to include these elements in the analysis one could either

use whole program analysis (very expensive - not feasible for a (incremental) compiler)

explicit contracts via an extended type system or annotations

The second option is well explored in research and some existing tools (like FindBugs)
already introduce specific annotations to this end.

One could argue that advanced analysis should be left to specialized tools (like FindBugs)
but having something like this in the JDT compiler should show two benefits:

feedback is more immediate and it is available for all JDT users without installing more software

analysis might be more precise than existing tools, because the actual flow analysis in the JDT compiler is already pretty strong (unproven claim).

Design Space for Inter-Procedural Null Analysis

This section discusses different options how tools for inter-procedural analysis could be designed. This write-up was made in preparation of designing a concrete strategy for the JDT.

Degree of Annotating

A radical approach would suggest that every reference type in the program must explicitly exclude or include the value null. E.g., String would not be legal type in any declaration (local variable, field, method signature) but only @NonNull String and @MaybeNull String are.

This radical approach has two problems:

it introduces vast efforts for annotating every type reference in the program

it is very difficult to apply to intermediate variables within a method body with branches, loops etc.

In the radical approach at least two more variable were necessary: at each point where analysis finds that a value cannot be null, a new variable with a differently annotated type would be needed. By contrast, the compiler can manage these intermediate states implicitly.

Thus it seems better feasible not to strive for full proofs of the absence of runtime errors, but to focus on gradually feeding more information into the analysis in order to just detect more (instead of all) potential runtime errors already during compilation. The radical approach can be weakened in two ways:

make annotations optional

limit the program locations where annotations should occur (note, how this actually relates to JSR 308).

In the vein of design by contract the following locations are relevant

method parameters (= method precondition)

method return value (= method postcondition)

fields (= invariant)

To truly reflect design by contract one might want to weaken the rules to

exclude non-API methods from contracts

butinter-procedural analysis is also relevant within a class, so annotating even private methods is useful, too.

accept inconsistent field states in the middle of a method body (only the first read and the last write within each method would be checked)

butsuch intermediate inconsistent states might be observed by concurrent executions and would make the analysis unreliable.

This is where JSR 305 comes into focus which covers the issue of standardizing "Annotations for Software Defect Detection"

After a first analysis I see two problems with JSR 305:

It has stalled, no official documents produced in 4 years, I couldn't find a proof of any activity during the last 2 years. JSR is marked as inactive, may soon be withdrawn.

It is far more generic than what we need for this specific issue: everything is built upon a meta annotation @TypeQualifier, it invests in supporting four states:

unspecified (no annotation)

@UnknownNullness (same interpretation as unspecified)

@Nonnull

@NullFeasible

The rationale for @UnknownNullness is for discarding an inherited specification. I wasn't convinced that any contract could be specialized to "no contract" - when specializing an inherited contract you should be explicit what the new contract is (which must be conform to the inherited contract).

Retention

Annotations have the advantage that a CLASS retention (or perhaps even RUNTIME) would support compiling against contracts in class files, which is not easily possible with the Javadoc based approach.

Standard vs. Configuration

Once the JDT compiler officially supports any specific syntax this creates a de-facto
standard which might conflict with existing and future standards (any code written against
the de-facto standard might be incompatible with future tools).

(1) doesn's look attractive to me. (2) is kind-of a workaround, carefully giving the message: we are not defining a standard, if you use this you should be prepared to change your annotations once a standard is created (automatic migration to a new standard shouldn't be so hard OTOH). (3) might be used to side-step the whole issue by saying: we're only doing technical exploration, but interested folks may still download and use this as early adopters. Technically, I would implement this using OT/Equinox :).
This would be an intermediate solution (still need to download additional stuff - but works as integral part of the incremental compiler) - shouldn't be difficult to migrate into the JDT/Core once we know more about standardization.

My irrational hopes are, that once users find out how great this is, someone will step forward and declare a standard. It would be great if people have something to play with and actually see the difference.

Semantical details

I see these issues worth discussing:

Do we need more than @Nonnull and @MaybeNull?

How exactly do annotations interact with inheritance (should mainly just apply the rules of design by contract, actually)

bug 186342#c14 points out that finding a good default is difficult because different not-annotated library functions have different contracts (e.g: HashMap.get() -> @MaybeNull, JTable.getSelectedColumns() -> @NonNull). Thus either default would produce a lot of false positives. For solving this issue one might use more fine grained control also over 3rd-party code, like a nullity-profile, with externalized defaults and exceptions / contracts.

Actually, a nullity-profile could also be used to map existing annotations within 3rd-party code to a different annotation class.

Actual Stragegy in the JDT

By default the JDT does not support inter-procedural null analysis, however, starting with bug 186342 the JDT can be configured to use annotations for null contracts across method boundaries.

Configuring Null Annotations for the JDT

By default the JDT does not recognize any null annotations but it can be configured to do so.
For this purpose three independent options exist:

specify the names of annotation types to be used for marking nullable vs. nonnull types.

emulate null annotation types that do not exist on the build path

implicitly import null annotation types in every Java file so that null annotations can directly be used by their simple name.

If at least one of these options is defined / enabled, the compiler will start analysing null contracts.

If null contracts are enabled ..

... without specifying annotation type names

→ the following built-in defaults will be used:

nullable = org.eclipse.jdt.annotation.Nullable

nonnull = org.eclipse.jdt.annotation.NonNull

... without enabling annotation type emulation

→ the specified annotation types have to be provided on the build path (either as source or binary files).

... without enabling implicit annotation type imports

→ annotation types have to be imported or referenced by their fully qualified name.

Null Contracts

Once properly configured the JDT compiler supports specification and checking of null contracts. Each part of a null contract implies an obligation for one side and a guarantee for the other side.

Method Parameters

When a method parameter is specified as nullable this defines the obligation for the method implementation to cope with null values without throwing NPE. Clients of such a method enjoy the guarantee that it is safe to call the method with a null value for the given parameter.

When a method parameter is specified as nonnull all clients are obliged to ensure that null will never be passed as the value for this parameter. Thus the method implementation may rely on the guarantee that null will never occur as the value for this parameter.

Method Returns

The situation is reversed for method returns. All four cases are summarized by the following table:

caller

method implementation

nullabel parameter

may safely pass null without checking

must check before dereference

nonnull parameter

must not pass null

may use without checks

nullable return

must check before dereference

can safely pass null

nonnull return

may use without check

must not return null

Local Variables

Null contracts can also be defined for local variables although this doesn't improve the
analysis by the compiler, because local variables can be fully analyzed without annotations, too. Here the main advantage of null annotations is in documenting intentions.

The following is an example of a program where all sides adhere to their respective part of the contract:

Althoug we know that toUpperCase() will never return null, the compiler does not know as long as java.lang.String does not specify null contracts. Therefor the compiler has to raise a warning that it has "insufficient nullness information" to guarantee contract adherence.

Null annotations for the local variables are redundant here, as the nullness information can be fully derived from the variable's initialization (and no further assignments exist).

Null Contract Inheritance

A method that overrides or implements a corresponding method of a super type (class or interface) inherits the full null contract. Its implementation will thus be checked against
this inherited contract. For the sake of safe polymorphic calls,
null contracts must be inherited without modification, or be redefined in the following ways:

A nonnull method parameter may be relaxed to a nullable parameter. The additional checks have to be performed in the body of the overriding method. Callers of the super type must still pass nonnull, while callers which are aware of the sub type may pass null.

A nullable method return may be tightened to a nonnull return. The additional checks must again be performed in the body of the overriding methods. Callers of the super type still have to check for null, whereas callers which are aware of the sub type may safely assume nonnull return values.

Any overrides that attempt to change a null contract in the opposite directions will raise a compile time error.

This explicitly implies that callers only need to inspect the null contract of the statically declared type of a call target to safely assume that all runtime call targets will all adhere (at least) to the contract as specified in the statically declared type, even if the runtime type of the call target is any sub type of the declared type.

Future

The following bugzillas address future improvements of the above strategy (order roughly by priority):