Building the right thing, building it right, fast

Who I am

My name is Jakub Holy and I’m a software craftsmanship enthusiast and a (mainly JVM-based) developer since ~ 2005, consultant, and occasionally a project manager, working currently with TeliaSonera AS in Norway. More under About and #opinion.

Posts Tagged ‘groovy’

There is plenty of generators that can produce numbers that are valid credit card numbers according to the Luhn check and specific rules of the individual issuer companies. However I have not found anything that would generate the missing digits given a bin, i.e. the first 6 digits of a credit card (the “bank identification number”). So I created one, reverse-engineering org.apache.commons.validator.routines.CreditCardValidator from common-validator 1.4:

Recommended Readings

Johannes Brodwall: This dependency injection madness must end! – it’s very valuable to hear well-founded arguments against any popular belief and Dependency Injection is one of these. “I have started disliking the consequence of this strategy very much: All coupling in my system becomes implicit and harder to understand. I have instead reverted to using design patterns like the Singleton pattern, but with a slight twist.”

Kent Beck: Functional TDD: A Clash of Cultures – TDD has been developed for object-oriented languages and applying it to a functional language with strong type brings interesting challenges. Also a good summary of the benefits of TDD: double checking of the logic (by the implementation and the, preferably quite different, test), solution decomposition (focus on part of the problem, once solve be sure it stays solved), automatic checking of correctness, outside in design (API first, implementation after that). Plus the pleasant experience of the continuous cycle of tension (failing test) – relief (green).

Tales from the Ops Side: Black Friday – an interesting and exciting view into the life of operations engineers one day when all went wrong. Key learnings: Many interdependant components are difficult to reason about; good monitoring and communication are crucial. The post refers to an interesting concept of Recovery-Oriented Computing, i.e. failures are inevitable and their prediction is nearly impossible thus we must focus on making the systems able to survive failures (e.g. vi damage containment, automatic fault detection, component-level restartability).

Groovy: The road map for the popular JVM language – why was Groovy created (as Java companion focused on productivity), key changes in Groovy 2.0 (more suport for static typing, Java 7, modularity with speed as a side-effect) and in the future Groovy 3.0 (invokedynamic everywhere, more Groovy written in itself).

You’re a Top Developer! – a surprising hypothesis that “90% of all developers never read a programming blog, never have any side projects to learn something new, and never spend any time or effort outside work hours to improve”. However I haven’t seen any data to back that up (the author only quotes Peopleware) and the author doesn’t propose any explanation for the fact. I’d really like to know if it is true and why it is so.

Business & Agile

Experimentation Is The New Planning – “You have no idea what’s going to happen to your industry. That’s why you build your organization into an engine of possibility.” We need “to continually develop options and explore possibilities” to survive in the ever-changing conditions. Successful strategies emerge from the many ongoing experiments. However, “For emergent strategy to be successful, there must be enough autonomy, freedom, and slack in the system for people and resources to connect in a peer-to-peer way”.

The Buy-vs-Build Shift (part 1) – Buy to reduce risk of failure (however true agile development – with frequent deliveries and feedback-driven direction – may be cheaper and more importantly can tailor the product to the actual needs) and to avoid inefficiecy of development (but it doesn’t need to be so with agile). “[..] in projects with long cycle times (years) there is a tendency for the business to be somewhat speculative and request all functionality that they can think of [..] With prioritised iterative delivery the business can halt a project when all features that are actually needed have been completed. [..] it does reduce the amount of features that are implemented, and based on my experience, quite substantially so.” Today’s development with e.g. TDD, powerful IDEs supporting automated refactoring, powerful development/production machine, the all-knowing Internet may be much more efficient.

European entrepreneurs – Les misérables – A good analysis of why it is much more difficult to be an entrepreneur in Europe than in USA (the strong negative impact of a business failure, lack of local investors, cost of firing people) and the decline of European entrepreneurship since 19th century/WW1.

Blackstag: Why Clojure? The author describes the set of reasons that have led him to Clojure – and those that actually made him stick with it. “[..] what I like the most about Clojure is that it brings together the best of what many languages have to offer while not forcing it all upon me and, in doing so, has provided a good balance between power and flexibility.” “With Clojure I accomplish more and have found a greater sense of happiness with the work I am doing.”

Gojko Adzic: How To Solve “Not Enough Time” – everybody in IT complains about too much work. The solution acording to Gojko? Kill software already made that isn’t needed (ex.: deleted 70% functionality that wasn’t used; remember that maintenance costs grow ~ exponentially with size/complexity). Kill software in making that won’t be needed (know the value – effect maps). Kill software that was not successful (if you measure the value of SW, you know whether its existence is worth it). Well written and inspiring. I’m looking forward to killing some SW🙂

Build The (USS) Enterprise – A fascinating site by an engineer who dares to think big yet manages to stay rooted in reality. A huge inspiration for us all! We tend to think too small. Having a great, inspiring goal is what moves us forward. And people are known to have achieve seemingly impossible things nobody else believed in (how many believed Columbus could reach India by by sailing the opposite direcetion? and even though he actually hasn’t, the impacts of his discovery were tremendous)

Hooking Users In 3 Steps: An Intro to Habit Testing – to be successful with a new web product in this age of distraction, you need your users to build the habit of using the app regularly (think of Twitter, Facebook, newspaper); this post describes how to find your habitual users, understand them, and optimize the application to turn more occassional users into habitual ones (identify habitual users – learn how they use the app & learn what turns random users into “devotees” – optimize the “habit path”)

Puppet: Serve configuration from a particular Git branch on demand (original: Git Workflow and Puppet Environments) – at Comoyo we use Puppet to configure all our environments and developers need to test their changes before pushing them to the live environment, preferably without interference from other devs. This post by Adrien of PuppetLabs describes how to enable each developer to have her private branch(es) and have Pupet Master serve the config from the branch on demand, using little puppet environments and hooks magic. (Notice that if using Puppet to serve files then you’ll need to have them inside a module, which is a good practice anyway.)

Speed, memory and LOC of Python 3 vs. Java 7 [fixed link] (the computer language benchmark game) – Python tends to take more time (though not terribly more) but is quite economical with memory and visibly more productive (3* less lines of code). Ruby is similar but way slower. Scala is little slower and more memory hunrgy but also more productive. Of course performance is rarely the key factor for picking a language on often it doesn’t matter that much.

Favourite Quotes

Goodhart’s Law: once a metric becomes a target, it loses its meaning as a measure.

Clojure Corner

What Sucks about Clojure…and Why You’ll Love It Anyway – 40 min talk by Clojure Programming author Chas Emerick. Some of the negative points: Namespaces – complex (use x refer etc.; no package.* import, …), no forward references; Dynamic scope (with-foo …) has subtle, complex behavior and hard to see => avoid when you can; Using STM (effectively) is hard even though it looks easy, it’s overused, think of it rather as in-memory db with superior integr. with Clojure (ex.: what is the right ref granularity? whole word map? small particle?), nondeterministic (which tx will fail/suceed? – can’t reason) => strange error under high load/mix of operations; JVM: long startup time, …; Macros look easy but are hard (don’t compose nicely with the rest); Function composition – hard to find failure root cause.

Why is Clojure so slow? (2/2012) – an interesting comparison of Clojure and other languages (C, F#, Groovy etc.) and an analysis of its slow start-up time (metadata and docstring building) and slower performance. (Interestingly, ClojureScript runs it 9* faster.) According to the author, it’s slower partly due to the imutable data structures (nothing new here, we knew we have to offer some performance for the increased safety and robustness). Conclusion: “Clojure is a beautiful, powerful and very useful language, but […] not great for small script-y programs.” Also, the usefulness of the benchmark is limited if you aren’t writing games. Plans for making Clojure leaner and faster are under way.

I needed a quick and simple way to enable some users to query a table and figured out that the easiest solution was to use an embedded, ligthweight HTTP server so that the users could type a URL in their browser and get the results. The question was, of course, which server is best for it. I’d like to summarize here the options I’ve discovered – including Gretty, Jetty, Restlet, Jersey and others – and their pros & cons together with complete examples for most of them. I’ve on purpose avoided various frameworks that might support this easily such as Grails because it didn’t feel really lightweight and I needed only a very simple, temporary application.

I used Groovy for its high productivity, especially regarding JDBC – with GSQL I needed only two lines to get the data from a DB in a user-friendly format.

My ideal solution would make it possible to start the server with support for HTTPS and authorization and declare handlers for URLs programatically, in a single file (Groovy script), in just few lines of code. (Very similar to the Gretty solution below + the security stuff.)

If you use the Grape’s @Grab annotation to get dependencies for your Groovy scripts at runtime and their retrieval fails with the exception “General error during conversion: Error grabbing Grapes — [unresolved dependency: …not found]” and a useless stack trace then you migth want to know that you can configure Ivy to log all the details of what is going on (what it is trying to download, where from, …), for example in the interactive groovysh shell:

Paul King had a very good presentation (last year’s slides) at JavaZone about why to use Domain-Specific Languages and how to create internal DSLs in Groovy. I’d like to list here few tips that he has mentioned but before we get to that, why would you want to create a DSL? Martin Fowler answers that in his Domain-Specific Languages book (2010). Some of the reasons are to have a higher-level, more focused and conscise representation that also domain experts can read and perhaps even write. You have certainly already used a DSL such as regular expressions, CSS, SQL, Spock‘s BDD tests, build instructions in Gradle – these are rather technical but sometimes DSLs are also created to be used by business users, f.ex. for anti-malaria drug resistance simulation. (Want more DSLs in Groovy?).

Paul mentions one important thing – you can always make your DSL better, i.e. more fail-proof (case insensitive, support plural endings, …) and secure and more like a natural language but it all comes at a cost and you must evaluate when the cost overweights the benefit (beware the 80:20 rule).

It’s sometimes useful for unit testing to be able to create a simple no-op stub of an interface the class under test depends upon and to intercept all calls to the stub, for example to remember all the calls and parameters so that you can later verify that they’ve been invoked as expected. Often you’d use something like Mockito and its verify method but if you’re writing unit tests in Groovy as I recommend then there is a simpler and in a way a more powerful alternative. Read the rest of this entry »

I like writing unit tests but Java doesn’t make it particularly easy. Especially if you need to create objects and object trees, transform objects for checking them etc. I miss a lot a conscise, powerful syntax, literals for regular expressions and collections, conscise, clojure-based methods for filtering and transforming collections, asserts providing more visibility into why they failed. But hey, who said I have to write tests in the same language as the production code?! I can use Groovy – with its syntax being ~ 100% Java + like thousand % more, optional usage of static/dynamic typing, closures, hundreds of utility methods added to the standard JDK classes and so on. Groovy support for example in IntelliJ IDEA (autocompletion, refactoring …) is very good so by using it you loose nothing and gain incredibly much. So I’ve decided that from now on I’ll only use Groovy for unit tests. And so far my experience with it was overwhelmingly positive (though few things are little more complicated by the positives more than compensate for them). Read on to find out why you should try it too.

(The arguments here focus on Groovy but I guess similar things could be said about JRuby, Scala etc. – with the exception of Java code compatibility, which you only get in Groovy.)

Few examples

Some of the example below use some Groovy magic but don’t be scared. You can write Groovy just as if it was Java and only learn and introduce its magic step by step as you need it.

(Of course this starts to pay of if either you don’t want to create a constructor or if there are “many” properties and you need to set different subsets of them (constructor with 4+ arguments is hard to read).)