Monthly Archives: July 2013

In my last blog post, The Case for Clojure, I discussed the advantages of using the Clojure programming language for writing applications on the Java Virtual Machine (JVM). Clojure’s success on the JVM led the language designers to compile the language for JavaScript engines, resulting in ClojureScript. Practically every device that can connect to the Internet has a web browser containing a JavaScript engine, making it an appealing target for developing applications.

Unfortunately, the JavaScript language contains flaws that make it difficult for developers to manage the complexity inherent in large applications. The widespread adoption of JavaScript presents a barrier to fixing these flaws without breaking existing applications. The JavaScript community has constructed various conventions in attempt to refine JavaScript’s faults, however these conventions have an ad-hoc nature and are not consistently applied. ClojureScript provides developers with an entirely new language and set of semantics that can be executed on any existing JavaScript engine. ClojureScript avoids JavaScript’s shortcomings while providing a number of new programming paradigms not currently available in the JavaScript ecosystem.

A Comparison of Clojure and ClojureScript

Clojure and ClojureScript are essentially identical languages, although they differ in the way that they deal with their respective execution environments, the JVM and the JavaScript engine. ClojureScript has the ability to interoperate with all of the interfaces that JavaScript uses, like the browser window, the document object model (DOM), and HTML5 APIs. ClojureScript uses the same immutable data structures as Clojure on the JVM. In a multi-threaded environment like the JVM, immutable data structures provide a nice alternative to the complex locking code used to preserve the integrity of data structures across multiple threads. Almost all JavaScript engines are single threaded, and only one particular section of code can run at any time. However, JavaScript engines are designed to handle asynchronous events that might be generated by a user interaction like a mouse click. It is hard to predict in what order related sections of code will run when developing JavaScript applications since these events can occur at any time. Immutable data structures make the development process easier by explicitly delineating when application data can be modified.

ClojureScript and JavaScript each share some common characteristics, so the difference between program structures is not as vast as the difference between Clojure and Java. ClojureScript and JavaScript are both dynamic, functional languages, so many concepts can be expressed with the same degree of conciseness. ClojureScript has an additional advantage over JavaScript, though, since it uses the same macro system provided by Clojure. This allows developers to extend the capabilities of the compiler. Well-designed macros allow repetitive, boilerplate code to be abstracted away, resulting in much more concise source code.

Namespaces

ClojureScript enforces a specific organizational structure for code called namespaces. All libraries outside of the core language must be referenced by their unique namespace (similar to Java packages). The lack of an equivalent structure can be a pain point for JavaScript developers. An interesting scenario involves the use of the popular JavaScript libraries jQuery and Prototype. Each of these libraries defines a global symbol ‘$’. JavaScript code using both of these libraries cannot differentiate between calls to ‘$’ in jQuery or Prototype without writing additional code to manage this symbol conflict. ClojureScript’s namespaces conveniently avoid such conflicts.

Development Tools

The ClojureScript community has contributed a number of tools to increase productivity during application development. The tool ‘cljsbuild’ leverages the Clojure build tool ‘lein’ to automatically download third-party dependencies and then compile ClojureScript code into JavaScript. It has a convenient feature to automatically recompile when it detects changes to the source code.

One of the most innovative development tools for ClojureScript is called the “browser REPL.” The acronym “REPL” stands for “Read-Evaluate-Print-Loop.” It is a concept that is found in a number of development languages and usually consists of a text prompt that accepts source code to be instantly evaluated. The browser REPL is an extension of this idea and will evaluate ClojureScript code in the context of a running web browser, skipping the usual compile/refresh process. While this is certainly a valuable tool, most major web browsers provide a similar console for evaluating JavaScript code. The ClojureScript community has taken this concept a step further and integrated the browser REPL with a number of popular text editors. This allows editing of currently executing ClojureScript source code directly in the browser from the comforts of a full-featured text editor. There are few other development environments that offer such a dynamic experience.

Language Innovation

The Clojure development community has a reputation for producing innovative new libraries to resolve issues across all types of problem domains. A recent example of this is a new library to do asynchronous programming called core.async. The main architect of the library (as well as the creator of Clojure) describes the library here. To summarize, JavaScript applications have traditionally used callback functions to handle asynchronous events like mouse clicks or responses from AJAX calls. This library has the potential to simplify how ClojureScript applications are structured, providing a nice alternative to callback functions. Due to the compiled nature of ClojureScript, in addition to the macro system’s ability to create abstractions out of portions of code, a library like core.async can provide new syntax without introducing any drastic changes to the language or the runtime environment. In contrast, the standard that JavaScript is based on, called ECMAScript, moves at a much slower pace in order to standardize any new syntax. A lot of the features available today in ClojureScript cannot be introduced in the ECMAScript standard until a large group of stakeholders agree upon them. This is often necessary due to the importance of stability in future JavaScript engines, but it often leaves JavaScript developers wanting for more powerful language features for constructing applications.

As browser applications become more complex, JavaScript developers must expend additional effort to produce correct, maintainable code. ClojureScript provides numerous features for dealing with application complexity and creating clear and concise code. I encourage readers to consider using ClojureScript for your next web-based project.

Searching the Internet for “information technology project failure rates” will provide a wealth of data and information. Data is readily available in depressingly large volumes from studies that indicate that investing in IT projects is high-risk and unwise. There is useful text accompanying the statistics that explains root cause and even classes of failures. I like these sites because the content is well organized and they include recommendations for how to avoid failure.

The Project Management Institute (PMI) was launched in 1984. The PMI material, certifications, frameworks (and everything else offered) are invaluable to any organization that runs an IT project. I applaud the PMI for giving the world of project management context and rigor to not only talk about improvement, but also a way to build methodologies to achieve the improvements.

Yet the statistics available from current studies, over 25 years after the PMI started helping us, are disastrous. According to the Calleam site listed above, a McKinsey & Company survey from 2012 showed that 17% of large scale IT projects fail so badly they threaten the existence of the company.

Again, there is no lack of material or training, even beyond what the PMI makes available, to help our IT industry improve. I often think of the maturity of the IT industry in comparison to the maturity of other industries like medicine, manufacturing, engineering, and construction. History indicates that assuming that customers will continue to invest due to lack of an alternative has turned industries and companies inside out once a suitable alternative becomes available. It is in our best interest to be in control of the revolution.

I know I’ve stepped into a complex dialog here, specifically with the subject of how to get the entire industry to move toward one common set of methods, practices, and cost structures. I argue the absence of common structure creates a gap that is filled by the client’s choice to bring unique and unrealistic expectations for cost, timing, and quality. Most consumers of IT services are well educated and know that writing a line of code only costs as much as the pay rate multiplied by the actual effort, which is not much. Yes, that is the wrong way to measure cost but it is the way that most consumers elect to measure it in a negotiation, because we as an industry lack structure. To compare, my recent trip to the doctor cost my insurers over $200, and I only saw the doctor for about 5 minutes. I’m sure that along with the 5 minutes I was also paying for a lot of staff, infrastructure, insurance, training, equipment, and capability that all came together to make the 5 minutes possible. And the doctor does not invoice the insurance company for the 5 minutes he spent, instead he associates it to a billable service with a preset fee. That approach allows him to be there for me when I need him.

In our IT world, individual projects are bound by unrealistic expectations from clients. Referring back to all the sound advice and training available from the above referenced sites, and from PMI, we know that project plans, which incorporate the boundaries of cost, quality, and schedule, must consider everything that is required to deliver an entire solution, not just the individual point in time that an engineer is writing a specific line of code. For example, that line of code, in order to be correct, must be written with an appropriate technology, within an architected solution that is secure, accessible, reliable, tested, and maintainable. Those adjectives are not free. They are in place because of hard work, training, and process.

The pillars of cost, quality, and schedule are non-negotiable once established. If a project fails, it failed because one or more of these three pillars cracked, or crumbled. Establishing them correctly up front will significantly reduce the opportunity for them to bear pressure during a project. Establishing them correctly requires that we manage expectations that include the teachings from PMI and from the lessons learned in the industry.

In summary, we need to plan for things like validation of requirements with stakeholders. We need to plan for things like training, performance testing, failover, risk management, system documentation, traceability… I could go on. My larger picture argument is that I hope that someday we’ll take this approach as an industry rather than as individual service providers. We should be differentiated by our ability to deliver rather than win business based on our ability to negotiate ourselves into low cost and inadequate delivery.

Set the pillars in place, build the plan based on the pillars, and deliver to the plan.

Please, share your thoughts and opinions on this topic and let’s see where the dialogue takes us.

Since the explosion of smartphone and mobile device technology spurred by the release of the iPhone in 2007 and Android-based devices starting in 2008, mobile devices are making more and more inroads into the enterprise business environment. Many major websites have released mobile versions that automatically adapt to the smaller screen real estate, and others have developed native applications that allow for even greater flexibility and function than is available on a web page. Smartphones and tablets are also being used as mobile point-of-sale devices, having completely replaced cash registers in some places. The future possibilities are nearly endless. However, integrating a new technology into an existing application infrastructure can be a challenge, especially for large enterprises, and in this post, we will discuss a basic strategy for adapting an existing enterprise application for use with a mobile interface.

Many large enterprises have invested in web application technologies, some homegrown and others off-the-shelf. While these applications are technically accessible from any device with a browser, adapting an application to a smaller interface can be a much bigger challenge than a simple website. Most applications have interactive editing features that can be difficult to navigate in a mobile interface. Creating a native application for these devices is arguably a better investment for the long run, especially considering that the enterprise has the luxury of dictating the specific platform to be used.

The environments for developing native mobile applications have evolved significantly, making the task of creating an interface for a mobile application a relatively easy one. The challenge then becomes the integration with back-end services. This is an area where the benefits of advancements in web services and service-oriented architecture (SOA) can really be seen. Enterprise applications that are already built on a service-oriented architecture will likely already have a web service interface to give clients access to data and application services. Also, development tools have advanced to the point that adding basic web services onto an existing server side application can be a relatively straightforward task. The key is to look at all mobile application interfaces in your enterprise simply as new consumers of these data and application services. If you carefully consider the specific use cases of the mobile interface you would like to create, it is likely you will be able to narrow down the required services to a relatively small number. It is therefore possible to roll out a whole new mobile interface to your existing infrastructure within a few short weeks or months, depending mostly on the complexity of the services rather than the mobile interface itself.

Finally, there is the question of how to interact with the web services from the point of view of the mobile client. With traditional SOAP web services this is a relatively easy task on the desktop due to the large number of development tools and libraries that provide most of the client code for you. In a mobile operating system, your options are much more limited. In the absence of a helping library, writing an interface to a SOAP web service can be time consuming and challenging. It is therefore a much simpler and more efficient approach to use architectures based on existing transport technologies and semantics. This is the environment where REST-based service architecture excels. Web services based on REST use basic HTTP and can pass data using simple existing formats like XML or JSON. In fact, there are a number of open-source libraries available to read and write JSON data, and more modern versions of mobile operating systems have actually included JSON libraries in their development API. With these tools in hand, calling a REST service becomes a simple act of sending an HTTP request to the right URL and making a simple call to read the JSON response.

On the server side, adding a REST layer over an existing service infrastructure is also relatively straightforward. Many of the existing web service application frameworks already contain features to make the addition of REST services very simple. Also, because the mechanics of calling the services are so simple, the process of testing the services in isolation can be as simple as opening a browser and hitting the right URL.

Obviously, the details can get more complicated than what I have outlined here, but in our business we have been able to prove that using this approach, a couple of developers can create a new mobile interface to an enterprise system in a matter of weeks rather than months.