Meta

Something I still love about Java

But recently I spent a day working on an old ~35k LOC Java code base, and it reminded me what I still love about Java.

The tools are magnificent

I was using Eclipse to refactor and modernise the code base. This involved stuff like:

Updating old data structures to use generics

Renaming packages

Updating to Java 1.7 syntax

Renaming functions and argument names to improve consistency and make them more self-documenting.

Looking back at the commits, I changed about 10k lines of code in about a day’s work. That’s roughly a line every four seconds. Certainly faster than I can type.

And the only reason I could do this was the strength of the tooling. The major time savers were:

Automatic warnings. Most importantly, this flagged which areas of the code base still needed work. A lot of refactoring is fairly brainless (e.g. conversion to generics), so I did whatever was needed to trigger them (e.g. changing a data structure to have a new generic type argument) and then used the warnings like a big “TODO” list.

“quick fixes” for may common warnings that I could just apply with a single mouse click. This helped tidy import statements, add generic type arguments etc.

Continuous syntax checking – this meant I could work very fast, in the confidence that mistakes would get picked up immediately by the syntax checker and highlighted in red

It was a long day’s work but at the end I was pretty stunned by what I had a achieved. A completely modernised code base, all tests passing and the warm glow of a job well done. I had even found and fixed a few subtle bugs and logic errors that my re-factoring work had exposed.

What was really surprising? Refactoring actually turned out to be fun. So much so that I even got carried away in my coding and turned up late to the pub. For anyone who knows me (or is at least vaguely familiar with the English affection for pubs….), that should tell you how seriously I must have been enjoying my refactoring :-)

Will dynamic languages ever match this?

Anyway, this experience led me to thinking when and if the tools in the Clojure world will get to this level of maturity and sophistication in terms of code management and manipulation.

Certainly, the Java world has had many years to mature and develop a comprehensive tool ecosystem. It shouldn’t be surprising that the tools are good, they’ve had plenty of years of widespread use and polishing. They’s had massive ongoing contributions from dedicated open source communities like the Eclipse Foundation and its related projects. There’s also the advantage of corporate use: the huge popularity of Java in the corporate world over the last 15 years has undoubtedly helped drive investment in tools.

Nevertheless, there are some things about Java that I think are intrinsic advantages in the tool space, more than just time to mature:

Static typing: this is clearly a big advantage: static types effectively provide an automatic sanity-check on every single line of code by asserting that you haven’t make a type error. While re-factoring, it helps you preserve an important invariant, namely that you haven’t changed the type of the inputs or outputs to any function. In a dynamic language you don’t have this assurance – ever passed a map to something that should have received a function in Clojure and wondered why you have a bug?

Compile time certainty: Java compilers can effectively build a complete model of the code at compile time – all the classes, methods are well-defined at compile time and this information can be used by the compiler to spot issues, provide warnings and make guarantees about the safety of many re-factoring operations. By contrast, it is much harder for a dynamic language to achieve this: the full model of the code only gets constructed at runtime (e.g. after you have run through all your “.clj” files, expanded macros and called all the relevant defines in Clojure). Thanks to our good old friend Turing we can prove it is impossible to fully reason about the overall structure of our code in such a situation.

A simple language – Java has a deliberately simple syntax – it was designed to be “Simple, Object-Oriented and Familiar“. As a Lisp, Clojure syntax is simple too of course – but in Clojure you can extend the language in arbitrary ways with macros, whereas in Java it is totally fixed. This guaranteed simplicity makes things much easier for tool writers. As a result Java simultaneously annoys programmers who wish they could do things like override operators while delighting the same programmers with awesome tools. Hard to have it both ways…..

Stability – admittedly this is more of a social phenomenon than a language features, but it is still important: the Java world has shown a very strong commitment to language and API stability, beyond that of most dynamic languages. You might argue that is a downside if you want innovative new language features, but again it is an significant advantage for tool makers and users who don’t have to deal with nearly as many version idiosyncrasies or breaking changes

For all these reasons, I am inclined to believe that it will be very hard for any modern dynamic language to catch up with Java in the many areas of the tooling space (by which I refer to areas like refactoring, automatic code quality analysis etc.)

Still there are compensations…

Nevertheless, I’m still going to stick with my Clojure. In particular there are two things that I get in Clojure that I think (mostly) compensate for losing the phenomenal IDE tooling that Java offers. These are:

The REPL – one of the things that truly makes Clojure coding great. I’d rather have the REPL at my fingertips and do proper interactive live coding than any number of static code analysis tools

Conciseness – so I may not have all the great Java refactoring tools. But if I need only 10% of the number of lines to do things, perhaps this doesn’t worry me that much. When I refactor a line of code in Clojure, it is more likely to be a genuine logic change than a mechanical change that is require to conform with syntactic formalities.

Finally this post wouldn’t be complete without a nod to the importance of testing. You should write tests regardless of whether you are using Clojure or Java. And it is particularly important if you are about to do a lot of refactoring that risks introducing careless errors or bugs. But I think it helps in dynamic languages a lot more: testing is an important way to compensate for the loss of the various compile-time guarantees that statically typed languages provide, especially around type safety.

Share this:

Like this:

Related

I concur a great deal with your post, but wrt C#. I create all of my personal projects based on Smalltalk, and more recently Common Lisp. C# has been my professional burden for the last 12 years or so (Java before that).

Sometimes, I find I miss some of the wizardry tooling available with Visual Studio (ReSharper specifically) when not at work, but maybe it is just the comfort you get from working with a net, even when you are only 5 feet off the ground.

Smalltalks mostly have nice refactoring tools available, but even they lack some of more powerfull context-aware refactorings available with static checked languages.

But I don’t find I need the more dramatic refactorings with CL, or maybe they are less usefull since I’ve re-learned search/replace from my old C days.

In the end, good tests are the main requirement to refactoring with confidence, regardless of the tooling available.

Hi and thanks for your comment. But I think I should put a couple of things straight.

Firstly, when did I say the refactoring was automatic? That definitely wasn’t the case – it took me a day of making real decisions. My point was that once I made a decision, the tooling did the rest of the work automatically, and it also conveniently highlighted places where I needed to make a decision (i.e. warnings).

Secondly, I think it is hard to describe code as “too low level” unless you actually know the context – sometimes “low level” is both necessary and appropriate. Especially if you are writing high-performance algorithm and data structure libraries (which is what this code base happened to be).

Interestingly some of the most “low level” code I have written recently was actually a Clojure-based code generator which used macros extensively – so absolutely would not be a candidate for automatic code analysis or refactoring. So I don’t think there is a correlation between being “low level” and ease of refactoring in any significant way.

I don’t buy the argument that you have to be statically-typed to benefit from compile-time type checking. For instance, in languages with HM type inference, you get all the benefits you mentioned above without littering your code with type declarations; the inference engine can just figure it out. What’s to prevent a similar type inference engine from operating on Clojure code that similarly lacks declarations?

Of course, it won’t work in the presence of things like eval and resolve, but you face the same problem in Java–the tools fall over in the presence of reflection. So I don’t think it’s an intrinsic advantage of Java; it’s simply a lack of maturity in the Clojure toolchain. Given a solid type inference engine, it would be trivial to implement a tool that could ensure that a given refactoring didn’t change the signatures of any key interface defns.

Hi Phil, I guess I’d count a language with full HM type inference as “statically typed” in the same way as Java…. i.e. my definition of “statically typed” is whether it is done at compile time or later.

Now I’d be the first to admit I’m not an expert in formal type system theory but I believe there are still a few things in Clojure/other Lisps that make compile time type checking into a hard/unsolved problem and not just a “we haven’t built the tools yet” thing – in particular I believe it’s harder than just applying HM type inference although I am hazy on the exact reasons.

Obviously I’d love a full type inference engine in Clojure! Typed Clojure in particular seems to be a very interesting project heading in that direction.

The main reason Eclipse and Visual Studio are so powerful is that many man centuries and millions of dollars were poured into them. If the same were to be poured into a Clojure IDE, the results would be even greater than Eclipse. Look at what’s happening with Light Table, for instance. It’s nowhere near Eclipse, but it also has been around only a very short time and it is already showing promise.

I agree with Phil. Static analysis is independent of the runtime semantics of the language. That means that you could do static type analysis (and other types of analysis) of dynamic Clojure code. Possible null pointer accesses, for instance, could automatically be detected.

I think the biggest challenge for Clojure IDEs, as you said, is that there is not fixed point of “compile time”. Each top level form can be compiled and recompiled independently. This is a challenge because the order matters, and often the order of compiling them and the order they appear in the file differs during development. This gap has to be bridged somehow. I often find myself clearing out namespaces and starting over loading files because I fear that the order of declarations has gotten out of whack.

However, this is also its greatest advantage. Having the code be “live” means the tools can actually just query the runtime. There was some talk a while ago (not sure if it is continuing) to have different versions of the Clojure compiler that add more information to the compilation so that it can be queried by tools. Eclipse has its own builtin compiler which does something similar for Java. So in Eclipse, Java is essentially live and dynamic. For Clojure, you could query the runtime for all references to a certain global Var, the location of the reference, and then the IDE could change the symbol refering to it. Similar things could happen with autocomplete, for example.

In the end, I don’t think that static languages have intrinsic tooling advantages over dynamic languages.

I certainly agree that Eclipse has clearly benefited from hundreds of man-years of work, and it would be great to see what could be done with a similar level of investment in Clojure IDEs.

I still think though that all the complexities you have outlined regarding order of declaration etc. make the problem of static analysis fundamentally harder – i.e. it is not “independent of the language”. Plus there is the simple fact that in Clojure people are more likely to use the kind of techniques that are virtually impossible to analyse effectively (stuff like eval, calling resolve on a constructed symbol, non-deterministic macro expansion etc.).

Ultimately, I think static languages will always have an intrinsic advantage *in this area* of code analysis and refactoring – that is not to say, of course, that dynamic languages won’t turn out to have much bigger tooling advantages in other areas – some evolution of LightTable working with live code being a good example of the possibilities.

Well written article and I concur with the author’s opinion.
I just started clojure and I find that there is no good IDE and documentation for the language. New languages should have ruby like IDEs and documentation…..should be crystal clear nd transparent…..

[…] of Clojure’s few weaknesses compared to Java relates to the difficulty of refactoring (see: Something I still love about Java). There are no static types to help you. You need to refactor tests. Tool support for refactoring […]

An answer to the tooling issue could be to eschew static analysis and static refactoring for dynamic versions of both. This might do things like splice ‘extractMethod’/’renameMethod’ into reflective calls; if the application includes a REPL and the user types in the deprecated old name, then the new name pops out. (Or in the extraction case, the whole new context and pattern for invocation.) A really clever implementation would also tell the user what the new lingo is and perhaps the rationale for change.

In your game world, global renaming of potions raises some of the same issues. (Particularly if potions can respond to game events, and so are, in effect, code…)

Your architecture of an immutable game state could help; if one approaches code refactoring from the same disciplined stance that the code is not allowed to actually be mutated either then it may be easier to address the interplay between the dynamic aspects of its execution and its evolution (by editing/refactoring).

As long as you can intercept and update the call before it happens, then you’ve achieved the refactor. A fool-proof way to do this might be to allow actual mutation only in the form of stubbing out the old implementation of the callee by a reflective call to refactor the caller. If the caller lies across a security boundary (for example, the caller is the human user through an (eval (read)), and we certainly lack the authority power to initiate brain surgery), then rewriting the caller might fail, but the dispatch to the new implementation could still proceed.

I think I’m coming to the conclusion that you can get the best of both worlds if you maintain a dynamic dependency graph at runtime – I can see refactoring of live code at the REPL being used to trigger recompilation of dependent code etc.