Measuring the Benefits of Ajax

October 6, 2005

There's a lot of hype surrounding the latest Web development craze, Ajax (Asynchronous JavaScript and XML), and a considerable amount of skepticism about its usefulness in the business realm. Surprisingly, although there is a lot of talk about what amazing things you can do with this approach, there is very little information about the applicability to business. There are quantifiable benefits to be realized for end users and businesses, which include improved usability and faster applications. These translate to hard cost savings that I'll explore in this article. It's important for developers to be able to explain the benefits of Ajax to other business stakeholders in these terms so that they are speaking the language of business, which is universally understood.

How Is Ajax Different?

For those who don't know, Ajax is a method of employing JavaScript, DHTML, and the XMLHttp behavior in the browser to provide truly dynamic content on a Web page without a page refresh. Popular examples of this technology include Google's Suggest, Amazon's Diamond Search tool, and many of Flickr.com's interactive features. Ajax effectively does away with the traditional "Click-and-Wait" Web-application architecture of yesterday, making it possible to provide the responsiveness and interactivity users expect from desktop applications. Ajax's ability to pull data from the server after the page has loaded contrasts with what we now refer to as the "traditional architecture." In a traditional architecture, the user must wait for the entire Web page to reload to see new results from the server. In an application that requires a lot of interactivity with the business layer sitting on the server, the user must reload the entire page many times. This has implications for the efficiency of workflow, the load placed on the server hosting the application, and the productivity of users.

Where Do the Benefits of Ajax Come From?

Often, in business, decision makers are interested mainly in how information technology can reduce costs, or make better use of information assets. The benefits of Ajax seem to come more out of the cost-containment arena than the latter. The question becomes "Where do these cost savings come from and how can we quantify them?"

1. Potentially Measurable Benefits

These are benefits that can be measured and expressed in terms of dollars and cents without much difficulty. Regardless of the quality of your Ajax UI, you will look to these metrics to estimate value. They include:

Time spent waiting for data to be transmitted: Time is money. Over many repetitions, the time employees spend waiting for the page to load can add up to significant costs.

Time spent completing a particular task: Increased efficiency in the user interface can often mean that time is saved at the task level, offering opportunities for concrete cost savings there.

Bandwidth consumed for the entire task: The cost of bandwidth does not increase linearly, but does increase as the company invests in larger-capacity Internet connections and new hardware to accommodate greater server loads. A firm's cost structure for bandwidth depends on the scale of their operation and these capital investment needs. That being said, the cost of bandwidth can be measured if this cost structure is known. If repetitious tasks consume a lot of bandwidth, these costs can escalate dramatically. The amount of bandwidth consumed also has implications for time savings.

2. Hard to Quantity Benefits

Some of the benefits associated with good user interfaces are qualitative and difficult to measure precisely. This isn't to imply they are not of financial value, but many business benefits are hard to quantify, and seasoned IT managers will know intuitively they can translate into significant bottom line savings. For Ajax, the opportunities for streamlining the interface are limited only by our imaginations, and it should also be noted that it's still possible to design a terrible UI with Ajax that doesn't benefit from any of the following:

Steps to complete a task: Reducing the number of steps has implications for the amount of time consumed but also for the number of opportunities for error. Fewer errors mean cost savings down the road when these errors would have to be manually corrected.

Familiar user interface: Quite often these days, Web-based applications are used to replace desktop applications that had superior user interfaces. The benefits of offering users a similar or even just a familiar user interface to what they use on the desktop means lower training costs, fewer errors, and greater initial productivity.

Improved application responsiveness: More responsive applications can improve productivity not just by reducing "wait," but by promoting a more fluid, uninterrupted workflow. In a responsive application, users can move rapidly from one action to another as quickly as they can visualize the workflow. Less responsive applications can defeat the user's workflow visualization by forcing them to continually wait for program information.

The Ajax Test Case

To explore the potential for cost savings, look at a sample application that was built to study the differences between the traditional Web architectures and Ajax. I call this application the "United Chemicals Sales Tool (UCST)." United Chemicals is a fictional company that sells household chemicals. The UCST allows sales personnel to transact sales and maintain a general database of customers. The application is written in ASP, and is connected to a database of roughly 10,000 customer records, 60,000 sales records, and a catalogue of about 600 products. This is meant to represent a controlled version of a scenario you might see in the real world.

Two versions of the UCST application were written. One version was developed using a traditional architecture. The user interface resembled one you might see if you had used the Microsoft Data grid to display and edit records. The other version used an Ajax component, the EBA Grid Control developed by eBusiness Applications (http://developer.ebusiness-apps.com) to edit data in an Excel-like interface. In almost every other respect, these applications function identically. For this to be a fair comparison, several guidelines were used in the execution of the test case:

Five test users were required to enter exactly the same amount of data in each application for each task.

The traditional architecture version would only post back to the server when absolutely necessary.

All HTML, JavaScript, and image overhead was included in the calculations with no pre-cached elements.

Test users received the same training in both versions of the application beforehand.

The United Chemicals Sales ToolAjax Version

Two tasks were designed to be representative of a sales entry tool. Task 1 was to add a sale to an existing customer. Users had to locate the customer from the list, and process a sale for that person. Task 2 was to add a sale to a new customer. This task involved more steps and required the user to enter the data, perform a search, and then enter the sale information. Both tasks together had 10 steps. Users performed both tasks on both versions of the UCST application.

Traditional vs. AjaxThe Results

The metrics taken for this study were bytes transferred, total time consumed by the task, and Microsoft Fiddler's estimation of the time it would take to transfer the same data in the same number of HTTP requests to a location on the US West Coast from our position.

As expected, in every test there were significant performance improvements across the board in the Ajax version. The greatest improvements were in bandwidth and network traffic efficiency gains. This resulted in improved responsiveness, allowing the user to move more quickly through the application. Overall, there was a 73% improvement in the number of bytes transferred in the Ajax application over the traditional version. This was due to the fact that that server requests were made only for the data that was needed, not for the entire page. Microsoft Fiddler also predicted that there would be a 68% overall time improvement in transferring that information to a location on the US West Coast given the number of HTTP requests, bytes transferred, the latency of that connection, and the bandwidth afforded by a simple 56k modem.

More importantly than bandwidth considerations was the direct benefit to users who saved on average 32% of the time required to complete the tasks in the Ajax version. Had those users been connecting to a remote location instead of a local server, Fiddler predicted the time savings could have been much more dramatic (around 68%). We can take these numbers now and extrapolate to see how large enterprises could be affected when these actions are repeated over and over.

The Benefit to Business

Using the test application with this 10-step process, one already can see that the potential for time and bandwidth savings is dramatic. It's possible also to express these savings in dollars and cents, given certain assumptions and using basic math. You calculate these savings based on the labor rate, the number of transactions, and the time savings per transaction.

Looking at a conservative potential time savings of 36 seconds per transaction, if a business performs 50,000 of these transactions per year, and a labor cost of $20/hour, we see a total labor savings of $10,000 per year, or 500 man hours, on this transaction type alone.

Given the Microsoft Fiddler predicted time savings of 199.01 seconds per transaction for a remotely hosted application, you see a more aggressive cost reduction of approximately $55,281 per year for this transaction type, or 2,764 man hours.

Looking at the Big Picture

To use the Return on Investment (ROI) view when making a business case for Ajax has advantages, one of which is being able to give some financial context to the many qualitative advantages of a more efficient web application. Taking this analysis a step further to gauge the long-term impact of bandwidth requirements would add to the understanding by being able to estimate the Total Cost of Ownership (TCO) when compared to an existing application.

Firms moving to Ajax tools such as the EBA Grid Control used in the test case have seen dramatic cost reductions in unexpected areas too. For example, consider opportunity costs. One firm was able to discontinue use of an expensive desktop application because the Grid allowed them to develop a Web application with comparable capabilities and without losing quality of user experience. The capital outlay of Web development projects is typically concentrated in up-front investment in human resources and licensing. However, the total cost of ownership can be much lower when you include the impact of yearly licensing on proprietary desktop software, as well as distribution and support costs for that application.

It might be short sighted, still, to rely on this kind of analysis to value Ajax. As mentioned previously, there are harder to quantify benefits that should be included in the decision making process. Not the least of these is the impact on employees. Better user interfaces can result in fewer human errors, lower training costs, and less overall frustration. Another avenue to explore is the potential to add new desktop-like functionality not possible in a traditional architecture. Ajax enables developers to provide new kinds of data visualizations and interactivity that would only have been seen before on the desktop, allowing users to do more with their information assets.

Conclusion

Although every new technology should be greeted with a healthy amount of skepticism, there are clearly demonstrable, quantifiable advantages to using an Ajax architecture in a Web application. These cost savings originate primarily from time savings, but also from reductions in bandwidth requirements. A representative test case showed that a business can save between 500 and 2,800 man hours per year on a 10-step hypothetical process, saving roughly 4 seconds per step (a between 30% and 70% reduction in labor costs). Although the benefits of improved application architecture extend beyond mere time savings, when included in the decision making process, an ROI approach such as this can help make a solid business case for Ajax.