CFML, Clojure, Software Design, Frameworks and more...

Real World Clojure - Prelude

October 24, 2011 ·

Clojure is still a fairly new language with a growing community that is very passionate about it. Understandably, folks new to Clojure - or still in the process of evaluating it - are concerned about how Clojure can be used in the "real world", to solve everyday problems that regular companies have. Alex Miller recently posted about how Clojure is used at Revelytix and whilst Alex provided the big picture, with both pros and cons, he didn't drill down into much detail.

At World Singles, we're using Clojure as a general purpose language for parts of the back end of our dating platform. Over a series of half a dozen or so blog posts, I plan to talk about how we're using Clojure, what problems we're solving with it and how we think it holds up. In full disclosure, World Singles has historically been a ColdFusion shop. Starting in 2009, a team - of which I later became part - created a new version of the dating platform, again in CFML (although this time on top of the free open source Railo engine, a JBoss community project). For a particularly troublesome part of the project, the various folks involved over the years had tried a number of solutions...

That particular problem was to constantly watch for changes to millions of member profiles, convert the updated profiles from relational data to XML, and post them to a search engine. I decided to try Scala and build a continuous process that could process blocks of profiles in parallel. Scala was appealing because it has XML literals, actors and the sort of functional support that would allow us to easily write and apply local transformations from result sets to XML. It worked pretty well. It performed better than what we had, but it suffered from memory leaks under load. Feedback from several people in the Scala community indicated this was a known potential problem with the standard actor implementation and we should look at Akka. Since we started on Scala 2.7 and had to live thru the instability of the migration to 2.8, it made us a bit gunshy about upgrades and migrations so we lived with it for a while.

In the meantime, I began to look at Clojure in early 2010. I hadn't touched a Lisp since the mid-80's so it was good to get reacquainted. It had a better functional story than Scala, it was much simpler - by design, based on Rich Hickey's principles - and it looked like it might be a better fit for my team, by virtue of behaving more like a dynamic scripting language... I decided to experiment by converting a few low-level pieces of our application over to Clojure - which went well - and then I did a spike to create a Clojure version of the Scala publishing daemon. I liked the result, we cleaned it up and put it in production.

Since then, we have increasingly applied the simplicity and power of Clojure to solve more and more problems that we had previously solved - fully, partially or unsuccessfully - in other languages. This series of blog posts will look at the pros and cons of using Clojure in our real world...