I have reviewed the book, Amazon SimpleDB Developer Guide by PACKT publisher, which got released last month. Amazon SimpleDB is a schema-less, highly scalable datastore which provides eventual consistency.

This book introduces developer’s to SimpleDB API & libraries in different languages, Java, PHP & Python. It discusses the differences between relational databases and SimpleDB, and how to deal with the No SQL, No Schema paradigm. It also dwells into profiling requests based on the execution times and box usage statistics, and adding cache layers using memcached.

Debugging exception stack traces is inevitable for any Java developer. When there is a failure at the customer end, our first reaction would be “Are there any exception stack traces?”. Stack traces play a key role for Java software to provide essential feedback on failures to the end user as well as the developer.

During the development process, there would be a lot of focus on exception handling mechanisms. The intention is to deduce or arrive at the cause for any failure with minimal effort. Nevertheless, there could be situations where you end up debugging or not able to debug bad stack traces which hardly give any information about the cause of the failure. In this post, I would like to focus on these situations and discuss possible solutions to fix such bad stack traces.

Dynamic proxy invocation is a well established pattern in Java. Especially, Containers that provide middle-ware services use this pattern extensively to intercept the calls and embed their services.

Stack traces while invoking Dynamic Proxies

Let us take a look at the invocation of dynamic Proxy instances and the stack traces of exceptions thrown during invocation.

In the above example, dynamic proxy instances are created for Customer and Administrator interfaces, and implementation for login() method is provided by the LoginProvider invocation handler. LoginProvider throws a LoginException when there is any failure during login, as shown below.

When you look at the above stack trace, first thing that comes into mind is “why did login fail?” and “who is trying to login?”. You can figure out why login has failed by looking at the exception message or the cause, but you can never get to know whose login has failed. In the above stack trace, you can see that the “$Proxy” masks information about Customer or Administrator interfaces, giving no clue on whose login has failed. Unless we have the source for LoginProvider we can’t deduce the problem, as we need to figure out the whereabouts of proxy’s creation and the interfaces it implements. Now, this becomes even harder if the interfaces implemented by the Proxy are read from configuration (Xml/Annotation etc.).

Tweaking exception stack traces

Many a times though there is enough flexibility in the programming language, we would hardly find a way to make use of it. One such a thing is modifying stack trace of an exception and we can explore this feature to fix our bad stack trace.

It is very simple to modify the stack trace, java.lang.Throwable provides getStackTrace() and setStackTrace() methods to get or set an array of StackTraceElements. So, you can create your own array of these StackTraceElements and set the array back on the Throwable instance. Each StackTraceElement can be instantiated and contain a customized class name, file name, line number etc.

publicvoid setStackTrace(StackTraceElements[] stackTrace) { … }

Understanding the above exception stack trace is really difficult for end users, system administrators or developers. Instead, if we can replace “$Proxy” with useful information like interface name, deployment name or JNDI name then we can quickly relate the problem to the component that has failed.

In the above stack trace proxy’s occurrence has been replaced with the interface name it implements. Now, you clearly know whose login has failed by looking at the stack trace. Someone can immediately point out that instead of tweaking the stack trace it would have been better to improve exception messaging and if required wrap the exception to contain relevant information. Yes, actually there is a trade-off between wrapping the cause and tweaking the exception stack trace. The trade-off with the tweaking approach is you can put up a generic stack trace manipulator without much knowledge about the exception. Where as in the other case we need to understand each type of exception and do the necessary wrapping.

Debuggers and Stack traces

Exceptions from a JSP page

How do you debug exceptions thrown when you access a JSP page? Aaahh..! Its a terrible pain to look into the generated servlet java files and relate the failure to the JSP line number. Some of the JSP containers support JSR-45, Java Debugging Support for Other Languages, to map the generated servlet java line numbers to the JSP line numbers and vice versa (through SMAP). This information is embedded into the servlet class file and is useful to debugger tools to track the line numbers of the JSP page back, when the servlet gets executed behind.

SMAP information is actually available to the JVM, but unfortunately VM doesn’t generate the stack trace based on this information. There is a bug reported for this issue and you can vote for this. Hence, JSP Containers need some way of transforming the exception stack traces to contain JSP line numbers, making it more readable to an end user. One of the ways could be the above suggested method to use setStackTrace() to post-process the exception stack trace after it has been raised.

Exceptions from obfuscated code

Another similar use case where we need a readable stack trace is, when the code is Obfuscated. If the obfuscation tools support JSR-45 and provide the obfuscated code to the original code line number mapping through SMAP, then we would have cleaner stack traces along with debugging support for obfuscated code.

I was going through Java discussion forums and happened to come across an interesting discussion related to Generics. It was about a loop hole in the Java Generics feature that allows a method to throw a Checked Exception without declaring it in the throws clause.

This is definitely going to be freaky and I wouldn’t encourage you to take advantage of this in your development. Initially I thought this could be a loop hole in the Compiler implementation, but later realized that the behavior is inherent with the Generics design.

Using the rethrow() method we can make java.lang.Exception to escape from main() without declaring it in its throws clause. The type <RuntimeException> specified while invoking the throwing() method is know as “explicit type argument”. (which is complementary to the type inference discussed below)

Type Inference

To understand how the above thing works, we need to get an idea about type inference and explicit typing. Take a look at the the following example

Surprisingly, both the store method invocations shown, compile. Compiler infers T to String for the first store() invocation as both the arguments are Strings, this process is known as “type inference”. We expect the second one to fail as T cannot be evaluated to both a String as well as an Integer. But, the Compiler infers T to a common super class or interface other than java.lang.Object. As String and Integer have Serializable and Comparable as super interfaces, Compiler deduces T to be a Serializable or Comparable and allows compilation. The result of which is an ArrayStoreException at runtime, defying the whole point of Generics and type safety.

How do we resolve this problem? We can use the “explicit type argument” in such cases.

The throwing() method defines type <T extends Throwable> which is declared in its throws clause and is no way related to the argument passed. The rethrow() method explicitly specifies the type as RuntimeException, making the method signature of the throwing() method to appear as shown below, at compile time.

Compiler evaluates the method signature by replacing the type variable T with its inferred or explicitly specified type, which is RuntimeException in our case. As a result of this Compiler doesn’t enforce exception handling or re-throwing it in the rethrow() method. After compilation and type erasure, all the generic type information is erased as shown below.

Some thoughts
Type inference or explicit type argument could do some damage when the Generic type information is not captured properly. In the loop hole case, since the exception argument passed to the throwing() method is not related to the Generic type (T), compiler was blind folded. If we modify the method signature as shown below, method invocation fails to compile.

Similarly, in the ArrayStore example we cannot expect the programmer to invoke the store method by explicitly specifying the type everytime at the calling context. A better way of capturing the generic type information in this case is to establish a relationship between Type variables(T’s), to enforce expected compile time checks.

publicstatic <T, S extends T> void store(T[] array, S item) { ... }

And interestingly, the following generic type declarations work as expected unlike the ArrayStore example.

Yes indeed it is really complex to understand type inference. All this appeals for better understanding of the type inference mechanism and how it applies. Probably you can get more information from FAQ by Angelika Langer.

In Java (prior to 5.0), a lot of times you are compelled to downcast your object to a more specific one. For example, when you add a String to a List, and when you want to retrieve your String back then you need to downcast.

Downcast is inevitable. Moreover, adding objects of any type to the list is allowed and the developer is responsible to remember the type of each object and perform the appropriate downcast while retrieving them. This gives a way to the type safety problems in Java, as every downcast in the code is a potential case for the wicked ClassCastException. Generics have been introduced to rescue us from these situations, they let you mark the collection to contain elements of a particular data type only, say a List of Strings. The syntax to specify a List of Strings would look like this. (Note: ‘type’ refers to any class definition in Java, e.g. String, Integer, Collection, MyClass etc. are all denoted as types.)

The syntax is fairly simple, you need to specify the ‘type’ of the Collection in the angular brackets following the Collection type. Such types are known as Generic types or Parameterized types. We’ll get to know more about defining collection types and substitution in the following sections.

With the above syntax, it is not allowed to add objects or retrieve objects of any other type other than String to the above List and doing that would result in a compile-time error. This is much better than the ClassCastException at runtime, and would definitely save a lot of your development time, isn’t it? And more importantly the downcast should be made extinct now, as the type of the elements within the collection is explicity informed to the compiler through the Generic syntax.

Purpose of Generics
Generics make your program well formed enabling the compiler to perform enough type checks based on the static type information provided and avoids unexpected type errors that could occur at runtime. Let us get into more practical matters.

Collections and Substitution rules
Collections are the primary motivation for Generics in Java.

Let us take a look at some substitution rules. The following are some legal assignments:

Here is the substitution principle for collections, (Rule 1) RHS of the assignment should contain a Collection implementation compatible with that of LHS and generified with the same type as that of LHS. The last assignment two assignments are valid, these are allowed to provide compatibility of non-generic (prior to Java 5) code with the new generic approach and vice versa. But, if you compile your code with -Xlint:unchecked option, the last assignment results in a unchecked conversion warning. (Rule 2) Do not ignore such compilation warnings, as they indicate your code to be unsafe (could break at runtime with ClassCastExceptions).

Does this compile? No, Iterator is not a generic type and hence the assignment of iterator’s element to the String ‘s’ (line 6) fails with a compilation error. Basically, in line 4 we lost the type information while obtaining the iterator and so we need an explicit cast here. Hence, you need to make the Iterator parameterized with String type to avoid the explicit cast.

Iterator<String> its = ls.listIterator();//4

(Rule 3) When you get an iterator, keySet, entrySet or values from a collection, assign to an appropriate parameterized type as shown above. This is because, these methods are modified to return their corresponding generic types to benifit no-cast code. Most of the Java 5 aware IDEs can do this job for you automatically, rely on them.

Though String is a subtype of Object, the second assignment is not allowed. Collection of Objects is a bigger set comprising of elements of various types (Strings, Integers, Cats, Dogs etc.), but a Collection of Strings strictly contains Strings and both of these cannot be equated (Rule 4). In programmatic sense if this were allowed, we would end up adding objects of any type to a List of Strings, defying the purpose of generics and hence this is not allowed.

Well, with the above restriction, how would you implement a method that accepts a collection of any type, iterate over it and print the elements? For such purposes, Wildcards are introduced for generic types to represent unknown collections.

Wildcards
We know that Object[] is the supertype of all arrays, similarly Collection<?> is the supertype of all generic collections which is pronounced as “Collection of unknown”. (Note: Collection<?> represents List<?>, ArrayList<?>, HashSet<?> etc. And Collection<?> is only a reference type and you cannot instantiate it, i.e. new ArrayList<?>() or new HashSet<?>() is not allowed.) (Rule 5) Collections parameterized with wildcards cannot be instantiated.

Using Collection<?> we can implement the iterate and print method as shown below.

Collection<?> appears very restrictive as you do not known the type information. When you obtain your elements from this collection you need to work with objects and would sometimes end up in explicit cast. So, strictly encourage Collection<?> when you need no type specific operations (Rule 7). But, there would be very few such use-cases in practice, where as more frequently you may need to perform operations on a base interface and you do not bother about the implementation type. In such cases you can benifit with the ‘bounded wildcards’.

Bounded wildcards
List<? extends Number> is an example of a bounded wildcard. This represents a homogenous List that contains elements that are subtypes of Number. Bounded wildcards only indicate an unknown type which is a subtype of Number.

So, we can obtain elements from the collection assuming the type to be a Number. But, you are not allowed to add anything to the collection as we do not know which subtype of Number the collection contains.

Differences between List<Number> and List<? extends Number>:

List<Number> is a heterogenous collection of Number objects (i.e. it can contain instances of Integer, Float, Long, etc.)

List<? extends Number> represents a homogenous collection of Number or its subtypes. It is instantiated with any of List<Integer>, List<Float> etc.

“? extends Type” is known as the upper bound, and we also have “? super Type” which is the lower bound where the unknown type denotes a super type of the specified Type. This is rarely useful with collections but could come handy when we define our own generic types.

We’ll see more about defining generic types, generic methods and type erasure semantics in my next posts.

In my earlier post on enums in Java 5, we have seen that static fields cannot be accessed within the enum constructor. With this restriction we could run into few initialization problems discussed below.

The other day Rajiv found it difficult to initialize a static cache during enum bootstrap. The following is his enum:

Rajiv says, “Deeps, check my implementation of getTag(int recordNo, int datasetNo). I would have liked to make a static map of (recordNo<<8 + datasetNo) vs Tag to make this method fast. Unfortunately, I cannot access that static map from the constructor.”

So, he would like to have a static cache of enums for easy retrieval based on their properties. But, the concern is where do we initialize this cache/map? In the discussion following the earlier post, we discussed that “enums are initialized before any static initializers are run”. Bang! We can initialize this cache in a static block as shown below (actually derived from Neal Gafter’s discussion on forum).

The new enum construct in Java 5.0 is used to define enumerations. Enumerations are not new to programming languages, they existed in C, C++, SmallTalk etc. In Java 5.0 they had to be introduced to overcome safety and ease of use concerns in using the earlier enumeration approaches. Let us take a look at how we used to define enumeration constants prior to Java 5.0.

Type Safety concern for enumerations

In programming sense, enumerations can be treated as just a sequence of integers. Let us define an enumeration that represents a sample of Colors (RED, BLUE, YELLOW, GREEN, ORANGE & PURPLE).

Whereever we need a color, we use the constants defined in Colors. But, what if an integer that is not defined for any of the constants is passed to the applyColor() method? Say box.applyColor(8), It fails. Shape.applyColor() method takes any integer, there is no "Type Safety".

So, it is not true to say enumerations are a sequence of integers. Enumerations should be treated as a separate type. Let us re-define the enumeration to take care of Type Safety.

Enumerated Pattern before Java 5

Prior to Java 5 release, enumerations were defined using the Enumerated pattern described in the book, Effective Java. Enumerated pattern takes care of type safety.

This enumerated pattern ensures that the Color type is Singleton and provides type safe enumerated constants RED, BLUE, YELLOW, GREEN etc. The toString() method is overriden to provide meaningful information when the constants are printed and the equals() method checks the reference equality of enumerated types, as these are singleton instances. Is this all enough to define a complete enumerated type? No.

Disadvantages of the above enumerated pattern

The enumerated pattern approach has the following drawbacks:

Enumerated types are not Serializable and Comparable by default. In case, if we make them Serializable,
Serialization of Color creates a new instance of the same Color. The equals() method fails to identify the equality of a serialized and a non-serialized Color. (Ofcourse, we can fix this by custom serialization of enumerated types. But, this leads to writing more boiler plate code for each enumerated type)

Color instances cannot be directly used in the switch-case statements, instead you need to use the color value

If a new color is added between existing constants or if the order of Colors is changed, clients must be
recompiled. (a.k.a Brittleness)

No support for iterating over the defined enumerated constants

Java 5 enum

enum provides a cleaner syntax to define enumerations in Java. And also overcomes all the drawbacks set by the earlier Enumerated pattern.

Adding new enum constants, does not require re-compilation of the client code

A static values() method is provided that returns an array of defined enumerated constants

Let us define the Color enumeration using Java 5 enum construct.

public enum Color {
RED, BLUE, YELLOW, GREEN, ORANGE, PURPLE
}

The enumerated types RED, BLUE, YELLOW etc. are of type Color. java.lang.Enum is the supertype of all enums and the Color enum implicitly extends java.lang.Enum. java.lang.Enum provides implementation for equals(), toString(), and hashCode() methods and it implements Serializable and Comparable interfaces. Enums are full-fledged java classes and can have arbitrary methods and fields. Enums can implement interfaces. Enums can have constructors and they take arguments in the declaration as shown below.

Each color has its own complementary color and to implement this we can use constant specific methods to return the complementary color. The method complementary() has to be either defined as an abstract method within Color enum or could be a method in one of the implemented interfaces.

Restrictions on enum types

Though enum types are full-fledged classes, the following are not allowed with enums:

enums cannot be subclassed

enums cannot have public constructor

enums are not allowed to be instantiated (using new)

enums cannot be cloned

All of these are enforced during compile-time.

Serialization of enums

Serialization mechanism ensures that new enum instances are not created on serialization and de-serialization (ensuring singleton behaviour). In a distributed environment when an enum is serialized from one JVM to another JVM, enum's state is not serialized. Hence, it is not encouraged to have state with enums which is updated by the applications. You can only have constant attributes that are initialized by the constructor.

Collections built for enums

java.util.EnumMap and java.util.EnumSet are the collections added for use with enum types. They provide a compact and efficient implementation for enum types. These collections are not synchronized and when used in multi-threaded environment, applications should take care of synchronizing the operations on the collection. EnumMap and EnumSet are homogenous collections of the same enum type. They cannot be used to operate on different enum types at the same time.

However, there are few things which could probably be improved in the coming versions of Java.

Currently null can be passed as a method parameter or assigned where an enum is expected. A compile-time verification for null enum assignment would probably avoid all those extra null checks that we need to incorporate in our code.

Static fields within enums cannot be accessed in its constructor. But, static methods are allowed to be invoked. The compiler enforces verification only for static fields and not methods.

However, enum constructors are invoked before static initialization is performed and hence the static fields would not be initialized. So, if you invoke static methods from the constructor which inturn access static fields, they would all be un-initialized.