Monthly Archives: July 2009

The Return On Investment for eCommerce Usability Will Always Beat Online Advertising, Because of the Principle of Amortized Improved Conversion

I don’t understand why online advertising agencies don’t add usability to their service offerings. Armed with a usability component, agencies could help improve the Return On Investment (ROI) for eCommerce companies well beyond any improvement gained by conducting online advertising. Why? Because of what I call the “Principle of Amortized Improved Conversion.”

Here’s how it works:

Let’s assume that you are a business owner and you have $10,000 you want to spend to help increase sales on your eCommerce web site. You can choose to spend the $10,000 for online advertising, which will generate incremental leads and sales. Or, you can choose to spend it on a usability improvement project on your eCommerce order form, which will increase conversion and sales.

Given this choice, which would you do?

If you’re smart, and I know you are because you’re reading this, you’ll pick usability – because you know an equal amount of money spent on usability will always beat the same amount of money spent on online advertising, given a long enough period of time.

This is because the one-time spend, and subsequent one-time incremental lift in leads and sales generated by online advertising is temporary. Turn on the advertising, you increase leads, turn off the advertising, and the number of leads go back down to their prior (normal) state.

With usability, the one-time spend in improving online form conversion creates a permanent increase in conversion. This means the cost of the usability improvement can be amortized over a very long period, which decreases the total cost per sale.

Here’s the proof, we’ll look at how online advertising, and usability, impact the sales and cost per sale of a web site over a one year period of time (but first, a few assumptions):

ASSUMPTIONS:
1. This eCommerce web site generates 2,000 leads per month naturally, with no advertising expense.
2. The conversion rate of leads into sales for this site is a nice 10%.
3. We’ll ignore “lights-on” costs associated with the sales on the website, it’s a cost of doing business and will not be included in the Cost Per Lead and Cost Per Sale metrics here.

Here’s what a typical month looks like for your pretend company:

You generate 2,000 leads per month through natural (not paid advertising) methods. Your 10% conversion in your online order form results in 200 sales per month. Well done.

The ROI of Online Advertising

In this scenario, you’ve spent $10,000 to conduct an online advertising campaign for one month. The good news is you’ve hired an excellent agency that provides an amazing campaign, and with your $10,000 you’ve generated 4,000 extra leads. That’s a total of 6,000 leads instead of the normal 2,000 for that month. That’s 3 TIMES (that’s 300% for you math wonks) the normal monthly flow of leads to your site. Very well done!

This provides you with an advertising Cost Per Lead of $2.50 (your $10,000 advertising cost divided by your incremental advertising-generated 4,000 leads).

Now let’s look at how this impacts your year (you may have to click on the graphic to see the full-size image if your eyes – like mine – are over the age of 40):

Now, let’s mash-up all the monthly metrics into a final yearly total. For the year, you had a total of:

28,000 total leads

$10,000 in online advertising cost

$0.36 total Cost Per Lead

10% conversion

2,800 total sales

$3.57 total cost per sale

The ROI of Usability

With this scenario, we start with the same assumptions. However, in this case you’ve chosen to spend the $10,000 on usability improvements of your eCommerce order form.

The good news is the usability work that took place over the first month resulted in a modest improvement on your site’s order form, resulting in a modest increase in conversion. Your conversion was 10%, but after the new user-friendly order form is launched in the 2nd month your conversion went up to 15%. This 15% conversion is permanent, and continues for as long as the order form (or web site) is active.

This is significant, it means that each and every month you’ll now be adding more sales, but with the same free leads, you were receiving prior to the usability improvements.

Here’s what the yearly total looks like (put your reading glasses on, or just click the graphic to see it larger).

Now, let’s once again mash-up all the monthly metrics into a final yearly total. For the year, you had a total of:

24,000 total leads

$10,000 in usability cost

$0.00 total Cost Per Lead

14.6% conversion

3,500 total sales

$2.86 total cost per sale

The Difference Between Online Advertising & Usability

So here’s what the difference looks like when we compare online advertising vs usability.

The usability improvement to the eCommerce order form resulted in an INCREASE of 25% more sales for the year, and a DECREASE of almost 20% in Cost Per Sale vs the Online Advertising scenario.

If we were to extend this into 2 or 3 years, the differences would become truly staggering, with that one-time usability improvement impressively beating the one-time advertising spend.

For those of you who are visual learners, here’s what these yearly differences look like in graph form:

I’ve actually been quite conservative with the above scenarios. The reality is I’ve very seldom seen an online advertising campaign generate 3 times the normal number of leads, and I’ve rarely seen usability improvements only increase eCommerce order flows by a paltry 5 percentage point lift.

In reality, usability beats the tar out of online advertising much better than I’ve indicated above.

As was demonstrated in the case above, over time, spending money on eCommerce usability improvements provides a much better return on investment for any business that wants to increase their web site sales, while at the same time decreasing the total cost per sale.

That’s the “Principle of Amortized Improved Conversion” in action.

Smart online advertising and web development agencies will add usability testing to their service offerings.

Smart eCommerce business owners will invest in usability improvements to permanently improve their sales and cost per sale, and thus their ROI of their web sites.

If you’d like more information about adding usability to an agency service offering, or conducting a usability optimization to improve eCommerce just contact me for usability testing information and I’ll be glad to explain how easy it is to get started.

There are 6 Easy but Powerful Tips for Job Seekers to Research a Company’s User Experience Culture. Prospective full-time or consultant applicants can use these tips to be better prepared when contacting a company.

I had recently asked on Twitter what usability topics I should blog about next. This reply from Jonathan Hung seems to me very timely, because I actually am investigating companies and their corporate user experience cultures right now.

I have been working in marketing, branding and user experience with large and small corporations since 1982, and during that time I’ve seen quite a few examples of how companies incorporate user experience in their culture.

I have grouped them into 5 models of corporate user experience cultures. These 5 models are based on where in the corporation the user experience practice resides, and what types of interactions the user experience practice has with the rest of the organization.

In Part I of this article, I reviewed the 5 corporate user experience cultures and provided a brief description of each. I included the pros and cons of the 5 models, as based on how the culture could best provide a business value to the company.

In this article, which I’ve decided to cleverly call “Part II,” I’ll explain the 6 tips I use to learn and research which of the user experience cultures a company may have – based from a the perspective of either a prospective consultant looking for a gig, or a job hunter looking for a full-time position.

For this article, I’ll use an example of a job that was recently posted on Yahoo’s HotJobs for a User Experience leader. This job posting, which was recently posted by Verizon Wireless, is a typical example of how I use the 6 tips to research the user experience culture.

The 6 easy but powerful tips to research a company’s user experience culture.

Tip 1 – Carefully Read the Job Posting Title & Reporting Division

In this example, we can see that the title is “Executive Director – User Experience” implying that this will be mostly a managerial position, with direct reports and responsibility to oversee the management of projects. Most likely, a good fit for this position is someone with prior experience managing user experience teams, which may mean little or no hands-on work. In addition, the reporting division is “Marketing.” This implies that the user experience group is the third of the five models, the UX in Marketing model. There are pros and cons to this model, note especially that political skills will be crucial, as often other divisions in a corporation may feel “Marketing” has little or no bearing on operations or I.T. UX projects.

Tip 2 – Read – In Order – The Job DutiesFrom years of experience creating job postings, I can tell you that almost always, the “critical” duties will be listed at the top, and the less critical duties will be listed in rank order underneath. Focus on the top 3 duties especially, as this is most likely where the company really needs help, and where you will be spending the majority of your time. Usually, the other duties are nice-to-haves or less important functions that occupy less focus, thus less time.

It’s very important to compare these top 3 job duties, and especially their order, with any Annual Report or other type of CEO communication which spells out the major goals for the Company. Misalignment between these top duties, and the Executive vision for future projects for the company spell trouble. I’ll review this in more detail below, but for now make sure you note the order of the duties.

In this example, note that the applicant will be required to have experience with mobile applications and devices. For the mentioned responsibilities, the top 3 in order are:

Note that the 2nd to last item is Identifying and driving cross-division synergies (this will be important later, after we review the CEO’s vision).

From the sound of the top 3 duties, it would appear that there is heavy need for team-building, unifying and only then researching and designing. From a user experience management perspective, it appears there are multiple areas in the company that must be aligned into a centralized user experience vision and team.

Tip 3 – Research Company NewsNow that you have a better understanding of the job and the top 3 duties, it’s time to research the context into which this job fits. Be sure to use Google or Yahoo News search to read news stories about the company, and also visit the Investor Relations or About Us section of the company’s website to see what news has been important to the company.

In this example, a recent merger of Verizon Wireless and Alltel as was announced in 2008 would seem to fit the pattern of the duties noted above, which were to build a centralized UX team and create a unifying vision of UX in the organization. After a merger, unifying diverse teams is very important if a centralized user experience team is going to be successful.

Tip 4 – Research the Annual ReportFor public companies, such as Verizon, the Annual Report is a goldmine of information. In it, the executives clearly define their vision or road-map for success.

Carefully note what the executives are defining as being critical to corporate success. If the job duties and descriptions align, then the user experience culture is in step with management and the potential for growth is there.

If however the duties and job description do not align with the executive vision, then proceed with caution. Misalignment of the user experience model with corporate vision spells an uncertain long-term future for the UX team.

In this example, the 2008 Verizon Annual Report has several statements that clearly define what the corporate vision is:

1. Move from voice and data to content & applications2. Move from separate platforms to unified platforms3. Build an application once then deliver it anytime, anywhere on any device

And later on in the Annual Report, the method for how the above strategic vision is to be accomplished is spelled out:

The good news here is these executive visions and methods closely align with the top 3 duties as mentioned in the job description.

More good news, notice that the executive vision includes “superior customer service experiences as a competitive differentiator.” Clearly, user (in this case customer) experience will have high visibility in the organization, and usability projects designed to improve the customer experience should have an executive champion, and thus executive support.

The only potential bad news, if any, is that there might not be the ability for the user experience team in this particular model to work with other divisions in the corporation on an equal footing. Referring back to the above job duties (noted by the arrow), note that the 2nd to last job duty is:

The fact that it’s 2nd to last might mean that it’s not very likely the user experience team will be conducting cross-division synergies, meaning the user experience model of user experience in Marketing might be somewhat of an island.

Tip 5 – Identify Compensation CriteriaIt seems more often than not that compensation is left off of many job postings. As a user experience professional, you must try to identify what compensation is available, to try to estimate if the job is a level on par with your experience.

In this case, it would appear from the above Annual Report that a bonus structure may be in place for this position that provides additional compensation besides just salary and benefits.

“Key objectives of our compensation programs are pay-for-performance and the alignment of executives and shareowners long-term interests.”

We could assume this means a bonus plus stock options / awards are part of the user experience compensation for this job posting.

Tip 6 – Use Your NetworkIt’s helpful now that you’ve done your homework about the position and company to use your network to try to establish contact with someone who works inside the company, preferably in the same division as the job.

Use LinkedIn, Facebook, Twitter and your other social media channels to try to contact an employee of the company. Often the employees may be able to provide you with much more information about the culture, the overall job expectations and importantly the stability of the company (especially in these trying times) than you can learn from other sources.

In addition, the internal contact might be able to put you in contact with the hiring manager, so that any general job questions you may have can be answered.

A word of caution here, resist the urge to contact the hiring manager and request an interview or meeting prior to the proper channels being used. It’s enough to gently probe about the position, not make a full frontal assault!

This overview of the 6 tips I use to try to identify the model of the user experience culture is not all-encompassing. There are plenty of other ways to learn more about a user experience model at a company. However, these are the 6 that I use most often, and hopefully they will help if you are in the position of researching a company, either for a consultant or full-time position.

If you have other tips that you use please post a comment below, that way we can all learn and grow smarter together!

I believe there are five general models of Corporate user experience cultures. Here’s a brief overview of each

I had recently asked on Twitter what usability topics I should blog about next. This reply from Jonathan Hung seems to me very timely, because I actually am investigating companies and their corporate user experience cultures right now.

I have been working in marketing, branding and user experience with large and small corporations since 1982, and during that time I’ve seen quite a few examples of how companies incorporate user experience in their culture.

What’s more, from the many consulting engagements and seminar / conference conversations I’ve had with user experience practitioners in other companies, both in the U.S. and abroad, I’ve learned that there seems to be a pattern to how companies incorporate user experience in their businesses.

These patterns are recurring, and can be found in almost all companies that have a user experience practice. I have grouped them into 5 models of how companies incorporate user experience in their culture.

These 5 models are based on where in the corporation the user experience practice resides, and what types of interactions the user experience practice has with the rest of the organization.

In this article, which is Part I of the two part series, I’ll review what the 5 models are, and then in my next article (which I’m cleverly calling “Part II”) I’ll explain how I learn which of the 5 user experience cultures a company may have.

Part I – The 5 Models of Corporate User Experience Culture:

Four of these models I see pretty often in businesses. However the 5th I’ve only seen rarely, even though I think from a business perspective, that it is the most powerful and influential of all the user experience models for business success.

So here’s the list which I have ordered in ascending order of benefit to the business – from weakest to strongest:

Model 1 – User Experience in I.T.

For many business cultures, the user experience practice is a component of the Information Technology division. There are several advantages to this, including incorporating usability as part of the Software Development Life Cycle in a very tight manner, and applying usability standards on all software that is created by the business.

However, I consider this the weakest model because of the strategic miss-alignment between the two functions of software production and user experience. Sounds crazy, right? After all, don’t we preach that usability and user-centered design must be present in software development? Yes!

But, I think the core strategic benefit an I.T. organization brings to a business is the ability to produce software on time, and on-budget, it is that division’s primary reason for being. I.T. executives and management are usually compensated based on their ability to deliver products when required. As such, typically a project plan with drop-dead dates usually rules all decisions. Yes, usability can and should be included in that project plan, and often is.

The truth is that I’ve witnessed many examples where even though problems with the application are revealed by the user experience team, the application process continues without many, or sometimes all, of the recommended changes being made.

Why? Because it would slow down the project and cause the deliverable dates to slip, which is the primary objective of I.T.. Thus the miss-alignment between I.T. and User Experience.

In my opinion, placing user experience into an I.T. organization, although beneficial from a process perspective, can cause the user experience to take a back seat to deliverable dates, and therefore is not as beneficial to a business as other user experience culture models.

Model 2 – User Experience in Operations

I’ve only worked for or consulted with a few companies that use this culture model of having user experience in the Operations division. But it can be a good way to incorporate user experience functions, especially for companies delivering Software As a Service, for which this is actually a fairly powerful model.

Operations is typically where the customer interactions take place, where the “rubber hits the road” so to speak. As many business owners can testify, saving (aka retaining) a customer is much cheaper than trying to acquire a new customer. So smart companies that include user experience functions in operations can continually optimize the customer experience, which means real bottom-line improvements in revenue for the business.

However, the problems inherent with placing user experience in an I.T. division are also present in this model. Because of the needs of operations, it’s possible that user experience issues might have to take a back seat to operational modifications that may help, or hurt, the user experience.

In addition, supporting operations from a user experience can be a very full plate, leaving little resource time to assist marketing or I.T. divisions.

Finally, because the user experience team reports to operations, other divisions may feel that the user experience function does not apply to them, and may either outsource or completely ignore the user experience needs they feel are necessary (or not) to support their division’s needs. This can often be witnessed for example by the Marketing team going to advertising or online agencies to create web sites without internal user experience support, or I.T. divisions that create applications with little or no user experience testing and optimization.

Model 3 – User Experience in Marketing

I have seen many examples of the culture of user experience being a function of a Marketing or Branding division. The advantage here is usability is often focused on driving better sales or transaction conversions, and thus directly benefits the business.

However, having had direct experience in this type of model, and having spoken with other usability practitioners who reported the same, I can tell you that there are significant detriments to this model as well.

First, because the usability function does not have direct influence over the other divisions, such as I.T. or Operations, often the usability function is considered strictly a “Marketing” function that does not apply to software development or customer service applications. The usability of a web site, especially marketing pages, is the normal realm of this model, but internally developed applications or customer service functions are often strictly “hands-off.”

Second, any conflict between the user experience needs and the needs of the other organizations are often handled “upstairs,” meaning the executive team must resolve the issue. Faced with either producing a product on time, and thus keeping investors happy, or modifying a project to suit a better user experience and potentially face delays, Executives will often choose the former, and thus user experience is not maximized.

Finally, Marketing budgets are normally the first to be slashed when the economy slows down, or when the company is faced with financial hardships. I have seen many organizations cut or eliminate user experience functions when these functions are attached to the Marketing division, because the user experience function is not considered a “core business function.” This negatively impacts a company’s ability to provide a superior user experience, especially when the economy is slower. This is bad for a business because when the economy is slower, it’s usually a golden opportunity for a smart business to increase market-share via a superior product.

Model 4 – User Experience as a Unique but Equal Entity

The business cultural model of having a separate, but equal, user experience division that services all divisions in the company makes sense, and seems to be a growing practice based on my experiences.

The benefits are clear, by reporting to none of the other divisions, user experience can operate in a non-biased atmosphere where equal resources are shared among the business units. In addition, the user experience team can operate as the holder of standards and best practices, and leverage learnings from one division across the other divisions.

However, the problem of “separate but equal” causes the same conflicts that occur in the other corporate user experience cultural models. Any differences between the user experience needs and the needs of the other divisions are often settled “upstairs” with Executives, who have the need to produce products on time.

For those of us in the United States, the term “separate but equal is inherently unequal” is a well-known phrase relating to Civil Rights and a famous Supreme Court decision to overturn the practice of separating people by race. Although less significant, the same phrase can be applied to the practice of having a separate user experience division as a business model.

The illustration above, showing the user experience division as being off to the side of the main business functions is sometimes an accurate display of the feelings of the business segment owners, who may feel that when pressed, their own division’s goals must come first over user experience goals.

As with the other models of corporate user experience culture, differences of opinion will often be settled “upstairs” by the Executives.

Because there is limited or no real control over the user experience, there may be fewer opportunities for the user experience to be maximized to the full extent possible across all divisions, causing missed improvement opportunities for the business and thus missed revenue enhancements.

Model 5 – User Experience as a Unique but Superior Entity

This is a rare corporate user experience culture, but examples are out there. If you replace “UX” in the above illustration with “Customer Experience” you’ll have an accurate description of certain Companies that take customers, and their satisfaction, very seriously (think Zappos, Google or Apple).

In some of these businesses, there is no actual user experience entity that oversees the other units (so remove the box in your head, or better yet, widen it so that all divisions are within the user experience box), but by providing incentives to the business division owners to continually improve the customer/user experience, the same goal is achieved – all divisions are focused on providing an optimal user experience.

In certain gaming and software companies, this corporate user experience culture drives all aspects of the business, which in turn drives continual optimization and improvements in the user experience.

Of all the models of user experience culture, this model by far can have the greatest impact on a business, because all divisions in the business are focused on maximizing the user/customer experience. Maximizing the user experience means more revenue and/or savings for the company, which over time adds to the profitability of the company, pulling it out ahead of it’s competitors.

So, that’s my vision for the 5 models of corporate user experience culture. Do you agree, disagree or have other models? Please share them in the comments.

Part II of this post will answer the question; “how do you determine what type of corporate user experience culture a company has?”

Among Dr. Mayhew’s many accomplishments, she is the author of several influential Usability books and has been applying her significant usability engineering expertise for Fortune 100 companies since 1986. Dr. Mayhew holds a Ph.D in Cognitive Psychology from Tufts University, an M.A. in Experimental Psychology from the University of Denver and a B.A. in Psychology from Brown University. Her books, “The Usability Engineering Lifecycle” and “Cost-Justifying Usability” are in my opinion two classics of Usability and User-Centered design.

1. What’s your background? Where did you go to school, what subjects interested you?

I majored in Psychology in college, discovering Cognitive Psychology late in my college years. Other subfields of psychology had interested me, but when I finally encountered Cognitive Psychology, I knew I had found my calling. What could be more fascinating than the internal workings of the human mind?!

I then applied to graduate school in Experimental Psychology, went to the University of Denver for my Masters where my initial Advisor was a researcher in intelligence testing, and finished my PhD at Tufts University.

My area of specialization was human problem solving, but I also studied general cognition, perception, developmental cognitive psychology, statistics, and neuro-psychology among other subtopics. My research assistantships involved work at the Colorado State Penitentiary doing IQ testing, and work in a primate lab tracking the social behaviors of macaque monkeys. Both my Masters and PhD theses were on topics in human problem solving.

2. How did you get into the usability field?

My graduate education was almost entirely funded by grant money. But one summer while at Tufts working towards my PhD, there was no grant money available for a stipend, so I needed to find a summer job to pay the rent. I had learned some programming skills in graduate school, so I looked for a programming job for the summer. I landed a job at a small high powered contract development and consulting firm in Cambridge, MA started up by some recent MIT graduates. This was 1978, and a PhD candidate in Psychology could easily get a job as a programmer.

At summer’s end, I liked it so much I decided to stay on. I got permission from Tufts to work full time while finishing my doctorate, which at that point, was mostly just completing my thesis. I worked for almost 4 years as a software developer and IT management consultant till I finished my PhD.

Then I took a job as probably one of the very earliest software usability professionals, at Honeywell Information Systems. That was in 1981. I was at the right place at the right time with the right set of dual backgrounds: software development and cognitive psychology. The field of software usability was just emerging.

I worked at Honeywell for a couple of years, then at Wang Laboratories a couple of years, and then went back to academia. I got a position as an assistant professor in the College of Computer Science at Northeastern University in Boston, where I spent a couple of years teaching usability courses to both undergrads and grad students. In 1986 I launched my consulting business, probably one of the first full time independent consultancies in software usability in the nation. I have been an independent consultant ever since, now in my 24th year.

3. What is it about usability that you most enjoy, or find most rewarding?

Although I have been working in the software industry since 1978, I still mainly consider myself a cognitive psychologist. I love learning and thinking about how the human mind works in the intellectual, information processing sense. In particular, human learning, memory, creativity and problem solving, as well as the more recent connections of those phenomenon to brain science, fascinate me. I enjoy the problem of applying what we know about cognition to the design of software user interfaces.

I enjoy all aspects of usability engineering – I like doing the research to understand different kinds of work people do and how they think about it and organize it. I really love doing user interface design. I also love applying what I learned about the scientific method in grad school to designing usability tests that will yield valid and useful results, and provide insight into how people approach software.

In addition, I love being a consultant because of the opportunity it provides to sample a lot of different industries, hardware/software platforms, application types and user types. The variety keeps it fresh for me.

I also love being independent. The only job as an employee I ever enjoyed was my first one as a programmer for a small firm of 50 people. Large corporations and academia were very frustrating work environments for me. Relative freedom from organizational politics and control of my lifestyle are very important to me. I live in a very rural area in a community I have been a part of since I was born, which is a lifestyle I love. Being independent means I can live where I want and how I want. The internet has meant I need to travel less and less to work with my clients, who are all over the nation and the world.

4. Your early research with usability and user-centered design methods (for example, “The Usability Engineering Lifecycle” from 1999) has been standard reading for anyone interested in an education in usability, how did you come up with this approach, what were your inspirations?

It’s a little hard to remember!

Certainly all my methodology was built on the work of others. I taught my first tutorial at CHI in 1986 (the very first CHI conference was in 1983), and the basic framework of my Usability Engineering Lifecycle was already there.

I think my main contribution with the 1999 book was synthesizing a lot of well established individual techniques into a lifecycle process. There is very little that is original in the techniques I describe in that book – what is unique is how they are all pulled together in a logical sequence where each one feeds into the next, as well as the templates, examples and war stories I included.

I never planned to write a book. I was teaching at Northeastern University in 1985 when a recruiter for Prentice-Hall came around my department hoping to talk professors into signing contracts to write text books. They put the idea in my head, and I started writing my first book, Principles and Guidelines for Software User Interface Design, which they published in 1992. I had originally planned for a single book to include both principles and methods, but the principles half got so big I decided to defer the methods for another book, which I then published in 1999.

As for the “Cost Justifying Usability” books, I had an “aha!” moment after reading an early article on the topic by Marilyn Mantei, then gave a talk at a conference on the topic. Randolph Bias attended that talk, and then decided the time was ripe for a book on this topic and contacted me to collaborate.

5. In 2001, you had argued that Discount User Experience methods would not sufficiently reduce project risk and insure Return On Investment. You had further stated that paying for Spot usability checks by “Gurus” would not either. In today’s environment, with AGILE software development processes, do you still hold to that belief, why or why not?

That is a very interesting question!

Yes, I do still hold that belief, perhaps even more so.

I view an application user interface as a coherent, unified system, that cannot be effectively designed piecemeal. Its like an automobile, a huge bridge, the human body, or any other complex system – all the parts have to work together in an integrated, smooth way. Thus, any approach that does not consider the whole is not going to be very effective.

I cannot look at a few individual functions, screens or pages and give effective advice on how to design them, without understanding the complete set of functionality across the whole application, not to mention a lot of key things about the users, the tasks they are doing, and the environment they are doing them in.

For instance, I cannot recommend using bold or a color as a cue for something like required fields, without knowing all the things that need to be consistently cued across the application and what cues I am going to use for what. Not to mention the fact that there needs to be a coherent information architecture that can only be designed based on an understanding of users’ tasks and the full set of functionality being offered in an application.

So for me, any approach that does not consider the entire application is going to fail to achieve consistency and coherence, the fundamentals of usability, and any approach that does not incorporate detailed information about users, tasks and environment is going to miss opportunities to tailor an interface to those requirements.

I think you truly get what you pay for.

There may be simple little page design issues that can be addressed without the big picture, but they are not going to buy you all that much. The most bang for the buck comes with higher level issues of information architecture and conceptual model design. So, the briefer and shallower the analysis, the higher the risk of missing key usability issues and opportunities.

As for Agile, I think there is an inherent conflict between that approach and sound usability engineering. I advocate a top down approach to UI design, that starts with the information architecture (in turn premised on an understanding of the full set of intended functionality and user tasks), and then a conceptual model design which is a set of standards for the visual presentation of and interaction with the IA, followed last by page detail design standards. That is, big picture established first, details to follow.

While I am not a student of the Agile method and have not actually practiced it (so far I have not worked with a client who has adopted it), I learned a little about it while consulting with a firm building software tools to support the Agile methodology.

While there is much about the Agile approach that is attractive (lots of iterations, feedback from the customer throughout, discovery of requirements as you go along via concrete prototypes), I think there is only so far you can get by putting a system together piecemeal, without considering the overall framework.

I wonder about the Agile approach because it seems to me that just like UI design, in system architecture design a top down approach is also most efficient and effective. The only way I can imagine the Agile approach working effectively is if a great deal of rework is done along the way as the discovery process evolves.

You cannot build a patchwork UI and have it be usable (or easily maintainable for that matter.) Thus, if you are going to approach it in a piecemeal way, there is a point at which you are going to have to throw out a lot of work and start from the top and design a coherent framework, in order to impose order on chaos.

One could certainly take an approach to UI design that would be compatible with the Agile philosophy and approach – I am just not convinced it will be either effective or efficient.

6. In your book “Cost Justifying Usability” you made the case that you must align your cost justification to the audience. How should a usability practitioner respond if the audience is a CFO, and that CFO expressly desires a “guaranteed” Return On Investment?

Great question!

And one I have thought about a fair amount. I have indeed been asked on occasion, and recently more often, to “guarantee” that my work will improve usability in some measurable way.

At first blush it seems like a perfectly reasonable request – I do after all claim that there can be a dramatic return on investment from expert usability work. However, the question is, how can the impact of usability work accurately be assessed?

Let’s consider an example. Imagine we are consulting on an e-commerce web site. The goal is increased sales, or “conversions.” A client might say to me, how about if you guarantee that your work will increase our conversion rate by at least 1% (which could be cost-justified, given my quote), or we will not pay. There are a number of problems here:

First, unless the client is willing to implement all of my redesign recommendations, exactly as specified, I cannot be held responsible for the results. This may seem reasonable to the client, but for many reasons – some technical, some political – they rarely do this. Who is to say if they have come close enough to implementing my recommendations for me to be held accountable for the results?

In many if not most of my consulting jobs, the client only takes some of my recommendations, or implements them in a somewhat different way than I specified. If this is the case, I cannot take responsibility for the results.

Second, many things impact conversion rates other than usability. For example, suppose a client took and completely implemented all my recommendations exactly . . . . just prior to the dot-com bust, or just prior to the recent recession? Who is to say whether it is the economy or my design that has impacted conversions?

Third, such a guarantee can work against my client too. What if they implement technical improvements which make their website work better on more browsers, or the economy picks up, at the same time they also implement my recommendations? Even if my recommendations are not responsible, other factors may raise conversion rates, and they would have to pay me regardless of whether I earned it or not.

It is rare that the only thing that changes from one release to another is the UI design. Usually many other things, both internal and external to the web site, change as well. In anything but a well controlled laboratory study, it is very hard to determine the sources of changes in measures of success or failure.

Fourth, if I could take controlled, laboratory-based before and after measures of usability in a project and use those as the measure of my contribution, that might at first blush seem reasonable. But, as we all know, UI design is part science and part art. Few of us could reasonably guarantee an improvement if not afforded the opportunity to iteratively test and redesign.

It’s rare that a client is willing to pay for multiple iterations of redesign and testing, or even before and after measures, so the criteria to determine if the the guarantee has been met or not is simply not cost effective in most cases.

In general, there are just too many other factors in play besides UI design that impact measures of usability. Thus, I don’t think its unreasonable to decline these sorts of requests for guarantees.

Finally, full time usability employees (or employees in any field for that matter) are not asked to guarantee a specific payoff from their work in order to get paid their salary, and I do not see any reason why consultants should either.

Unlike employees, clients simply do not have to rehire us if they are not happy with our work. They generally have to live with poorly performing employees. Consultants survive (or not) by establishing credibility in other ways – word of mouth, references, publishing well regarded books, etc.

What I offer my potential clients instead of guarantees is success stories.

For example, a large government client from a couple of years ago recently contacted me to work on a new release of an intranet I had redesigned for them previously. They had implemented most of my previous recommendations, and told me that the new design had been a big success:

User complaints had dropped from 10% to 0%. Many more users had adopted the intranet as a work tool. They had received an internal award for innovation. Their intranet was now considered an example of best practices in usable design within their agency. They found they had much more clout in recruiting other internal development projects to adhere to some of their standards.

When you can cite stories like this, and give references from clients who have benefited from your work in these ways, this is good enough to be expected to be paid for your time and expertise.

7. Social Media, with its almost instant communications threshold has seen rampant growth in the past few years. How are the Social web and instant connected communications like Twitter, FaceBook, Instant Messaging and the like changing usability and usability testing, or are they?

I wouldn’t say social media is actually changing usability or usability testing – they are just new types of applications, where all the same old principles and methods of usability still apply.

When the web first came along, people said the same thing, they seemed to think everything we had learned about usability in the context of traditional desktop applications had suddenly gone out the window, and we had to start all over again.

It just was not true.

People have not changed much in the last 50 years (or 50,000 for that matter) and all of the principles of usability are premised on understanding human information processing capabilities and weaknesses. Usability methods are similarly premised on how to measure and interpret human behavior.

While changes in technology do open up new possibilities, or place new constraints on the design of human-computer interaction (eg, limits of handheld mobile devices), most of the design principles and methods are universal and platform-independent.

For example, human memory constraints, and the advantages of consistency, are the same regardless of whether you are designing for a large screen, or a handheld device – you just have to apply the principles to different technological constraints and capabilities. Similarly, testing a design on either type of platform will follow a very similar if not identical methodology.

What I do find interesting about social media in the context of the business world is the opportunity to consider incorporating the concept of social media into corporate software to build community, share corporate knowledge assets more effectively and promote professional networking for the organization’s benefit.

I have just been helping a client consider how to use these new concepts to support their business goals through their intranet, and the possibilities seem to have a lot of potential. There will certainly be usability issues in designing these capabilities and usability testing will be just as important as ever. There is just not a whole lot of new principles to be revealed or methodologies to be devised. The same tried and true usability approaches will work just fine.

By the way, my opinion (and the opinion of many of my peers) is that Facebook has an abysmal user interface. They keep changing it, but apparently not based on any usability science, as it remains flawed, just in different ways with each modification.

Social media applications need usability input, just as any other type of software application.

If someone produced an application with the same functionality and a superior user experience, Facebook would be in trouble, just as IBM was in trouble when the Apple Macintosh came out.

8. In your opinion, what’s next for usability? What should usability practitioners and web site designers be focusing on for the future, say the next 2-3 years?

I think designing good UIs for small mobile/handheld devices is going to be a big area for our field, and applying social media in ways like I just described above as well.

In addition, I think we need to welcome and learn to work effectively with other professions and skillsets required to create a truly optimal User Experience: graphic designers, eCommerce “persuasion” experts, SEO experts, pay-per-click experts, etc.

The most powerful and successful user experiences require all these elements (and probably more), and it’s amazing what we can create working in interdisciplinary teams, that none of us could produce alone.

9. What are your plans for the future? What are you looking forward to doing next in your career?

Well, actually, I am working on a very exciting new project which I cannot talk about yet. Ask me again in about 6 months!

Thank you Dr. Deborah Mayhew!

It’s refreshing to hear that even with all the changes in terms of communications and the near-instant aspect of the new Social Media, the same basics of good usability and user-centered design apply.