We started this website in late August, 18 weeks ago. It’s a learning experiment. We were two buddies, both web designers, who wanted to know more about web usability and user testing. We gave ourselves the goal of learning what we could by December 15th.

By my calendar, that’s tomorrow.

It’s also nearing the end of the year. And the end of the year is always a time for reflection and for taking stock.

Since we’re at the end of our original learning period, Newman and I felt it was only right to look back on what we’ve learned to see what the key takeaways have been.

We created a list of our Top 10 Key Concepts we feel we learned about building better websites.

This is a decent chunk of material to get through so we’ll be covering this topic for the next three posts. Today, numbers 10-6. On Friday, numbers 5-1. And on Monday, a podcast where we hash it all out. It’ll be fun!

10. Proactive vs. Reactive

Proactive > Reactive

The natural state of things is to be reactive. Reacting will save your skin if you are trying to run from a giant tiger leaping out from the jungle. In our modern times, troubling as they may be, there are no tigers. Death isn’t constantly at the door (and if it is, you’re doing it wrong).

Reacting has a down side though: the success of the reactor depends on the actor.

For a business owner, or really, any goal oriented individual, it’s a bad place to be.

Oh, it’s a comfortable place to be, but it’s not really how you want to be. Nobody has ever gotten their way because they were reactive. You don’t lose weight by not worrying about it. You don’t grow your business by only taking the business that comes through the door. It’s not a long-term strategy.

The advantage of being proactive is that you get to actually have a strategy. It’s chess, not checkers.

We talked about this as it relates to the small business owner back in September where we described an ideal initial meeting and how to create valuable monthly reports. This is important for UX designers because testing is inherently a proactive thing. It doesn’t happen in a reactive system. This is why oftentimes a cold wind starts to blow when the topic of user testing comes up. And it’s also the next key concept we ran into:

9. UX Testing Resistance

When told clients and business owners we know about UX testing, they generally respond negatively. “Watch people use our site and ask them stuff? Nah, I think the site is fine.” they would say. While we didn’t expect them to jump for joy, we did expect a little more… interest.

Why did they react this way? It’s because oftentimes it’s being introduced into a reactive environment and it rubs the wrong way.

I grew up playing the drums and I once got a piece of advice about how to play. The idea is that a drummer typically plays a little behind the beat or a little ahead of the beat. And either one is okay to do. The one thing you can’t do as a drummer is to shift from playing behind the beat to ahead of the beat mid-song.

That’s what introducing a proactive element into a reactive system is like. And its what accounts for a large amount of a business owner’s gut resistance to it. It doesn’t fit their work model.

And for the ones that it does, it’s still a matter of getting it incorporated into their system. So it’s a bit of an uphill battle, at least initially, unless you have forward-thinking clients.

Like most everything, the proof is in the pudding, and most businesses come around once they realize that user testing is a key component in raising their web site’s revenue.

8. The Power of Process

What's your mental image of ultimate power? This is mine.

User testing fits best inside a larger structure of improvement and goal setting and seeking. It practically begs for it. The reason somebody would do a user test is because they plan on iterating their website. And once a website has been iterated, the change – good or bad – can be measured.

And that is essentially all the process you need. Once it becomes evident that sales/leads/revenue increases when improvements are made to the website and that the improvements are discovered through user testing – goal setting and allocating a proper budget for growing the web business will follow. The process will emerge. It has to. It’s how we make the website a better producer.

7. The Difference Between User Experience and Usability

User Experience Design is the umbrella term for many difference fields, including usability

It seems every sub-culture or community has weird semantic issues. When it comes to naming things, people aren’t very good. We like to borrow words that aren’t clear and then use them indiscriminately. As we jumped into the world of UX – which is short-hand for User Experience – we discovered a whole new vocabulary. One of the keys was understanding that User Experience was the umbrella, the super-system which includes Usability.

Usability is the technical side of UX. It is quantifiable and measured with tools like google analytics and A/B tests. User experience includes the qualitative and holistic experience. It’s harder to measure and includes emotions and feelings of the users. It is mysterious and an inexact science, but doing the ux techniques can reveal mistakes in a web design and improve sites.

6. The Critical Path

Two Critical Paths! I found it... wait a minute. PHOTOSHOP!

We first talked about the critical path back in late September in the post “Cut the Chute and Get on the Critical Path (to Profit)”. The concept is an easy one: design your website so that all of the energy is about moving a user from the beginning of the process to the end of the process. Define the pages in the critical process and boom, there’s your critical path.

But more important for me was the ability to finally know what’s supposed to go on a website.

That sounds like a shameful thing to admit: a web designer doesn’t know what’s supposed to go on a web site. But as a designer, we get wrapped up in the technical aspects of what we do. When we nerd out on our thing, we argue about technical arcana. For nerds, it’s not about solving business problems. That’s why there are business guys.

But business guys don’t know either. I’ve been building websites for a long time and it’s rare that a business owner takes ownership over the content and ordering of that content on the website.

They just leave it up to the nerds, and at least in the local market, nobody seems to specialize in knowing what goes where.

So the critical path is a big f-ing deal. Now we have a way to lay everything out: Do it in the way that makes the most sense to achieve the objective of the website.

This week’s Podcast has a special guest, Paul Veugen, who comes to us via Skype from the Usabilla Headquarters in Amsterdam Holland. Conducting the interview I was a bit nervous – I stumbled, said ‘um’ too many times, but regardless of that, we had a pretty good time. I think you’ll enjoy it.

We talked about the tool and how it can be used, naturally. But we also discussed, what I feel makes Usabilla stand out – it’s organizational culture. It’s a relatively new company,a start-up. Paul humbly describes it in a matter of fact tone. He needed a tool for his university studies and cobbled it together. Then, a friend convinced him to build it as a scalable app and they started a company. He got some clients and then got some external funding. “See, it’s fuggin’ simple”, says the Tenacious D frontman Jack Black. The Usabilla team’s creativity is evident in their use of social media – in blog posts, updates, and interesting uses of the tools – like a report created for the wider community on UX on eCommerce Sites. They get it and it shows.

It’s not so much what they do (usability testing) but how they do it. Plainly stated, I think they are having fun. And, fun is contagious. So, thank you Paul and the Usabilla gang, for sharing with us. Good luck and Enjoy!

Ben and I got to discussion a post on UXMovement.com titled “Are You Meeting the User Experience Hierarchy of Needs?” by Anthony. It was interesting and had a bunch of comments. I decided to research it more and share it with you. here:

First Maslow’s

Pretty much everything from the 40s sucked... name one, really

Everyone ‘sort of’ understands Maslow’s pyramid. They may not understand it fully. It may be like Darwin’s theory of evolution – everyone can give you the basic tenants, but it is probably misunderstood. Here’s my understanding of Maslow’s hierarchy of needs. Food and water and shelter are basic needs that take precedence over things like morality and respect of others. You must satisfy the basic, base levels of the pyramid BEFORE you can / want to satisfy the higher levels. Maslow’s hierarchy of needs is often cited when explaining why a typically law-abiding person will break the law (higher order) in order to get food or sex (lower order). It’s a theory I have problems with.

Here are my questions: In order to be a whole person do you need to satisfy all needs? Is it harder to meet higher, mid, or lower needs? Does the pyramid establish a sequential order for achieving needs? And, to transition to interface design, must a “quality” website satisfy the all needs? Is it harder to meet the higher needs than the lower needs? Must the design process start with satisfying lower basic needs and move on to higher needs?

This article and chart puts functionality as the base of the pyramid. Together with information, this forms the first half of basic needs. The top half are higher needs with aesthetics below and usability on top.

Usability breaks out of the pyramid top suggesting that it is not inside the pyramid – or this could have been done to promote readability as the font would be very small if made to fit in the tip of the pyramid.

He outlines a model that says you must begin interface design with functionality and move on to content and aesthetics in order to achieve usability. Anthony defines usability as “the ease-of-use of an interface that increases user productivity.” There was some debate on this article and several commenters wanted to put aesthetics at the top of the pyramid – which makes me scratch my head.

Even though Anthony did not like this pyramid Chart, I like it quite a bit. Anthony called it pretty bullshit (which I thought was slightly rude). Just like a cow patty, this chart can teach a lesson. Working from the outside in, the chart establishes two contexts. At the top we have subjective/qualitative and ‘focused on experiences( people, activities, context)’ with an arrow pointing downward. The second context is at the bottom it states: objective/quantifiable ( products, features) and ‘focused on tasks’ with an arrow pointing upward.

Within the context of tasks (going up), the graphic seems to relate that the priority sequence is: functional( useful), reliable, … and peaking at meaningful.

Within the context of experience (going down), the graphic seems to relate that the priority of sequence is reversed – that meaningful is the first priority and functional useful is the last priority.

This chart tries to make the distinction between subjective experiences and objective tasks. That’s what I like about it. It seems state that if a subjective experience is meaningful and pleasurable that it does not have to be reliable or functional. Or that a subjective experience that is meaningful and pleasurable need not be reliable or functional. And this DOES seem to be descriptive of my personal experience.

If you have a meaningful and pleasurable relationship with a beautiful girlfriend, you don’t really care if it’s reliable or functional [Radio Edit]. But I don’t think he’s talking about relationships. If you just love your iPhone , you can subjectively overlook usability [Siri, why can’t I just drag and drop my files from my computer to my iPhone. Siri?] – Think supercars in the 80s – Toyota’s were WAY more usable. If you were focused on tasks, then you just care if it’s useful and reliable (Toyota)-you don’t really care that it’s not meaningful or pleasurable (Ferrari).

The fact that convenient lies in the center of hypercatalecta‘s diagram is important. That says that convenience is of equal value for objective users and subjective users. Both those focused on experiences and those focused on tasks care about convenience.

But here’s my problem: why is it a pyramid?. I have a feeling they just did that to make it look pretty. Perhaps this is why Anthony called it pretty bullshit (He called another guy pretentious ) But, because it defines the subjective / objective vectors, it gives me valuable insights. When working with some clients – subjectively oriented clients – they really only care about aesthetics… but here is where I think folks are missing the point – the reason they feel that way, is because that’s their experience. That’s how they experience the site. It’s their user experience.

This is why I concur with Paul Veugen – and others – that user experience is a super-system of usability. And, user experience is a complex interaction of many facets. Why? That little word USER. People are complex – they are different shapes and sizes, speak different languages, and use websites in different ways and for different reasons. Simply put – WE are the definition of complexity on this planet …currently

Alternatively, usability is technical. Usability can be measured… and can be measured by any asshole. Can you hear me now?

Voted most popular (non-contexual) user tester!

Moving forward –

Here’s a slideshare from Lyndon Cerejo from back in 2001. This is actually a very good article and I need to spend more time with it… but I’m getting pretty sidetracked here.

Lyndon Cerejo's insightful pyramid from a decade ago

Ugh… let’s look to Smashing Magazine, surely they are shed some light here.

Based on Maslow’s hierarchy of needs, the idea of a design hierarchy of needs rests on the assumption that in order to be successful, a design must meet basic needs before it can satisfy higher-level needs. Before a design can “Wow” us, it must work as intended. It must meet some minimal need or nothing else will really matter.

Is this true? Or could a design that’s hard to use still succeed because it makes users more proficient or meets certain creative needs? Do you have to get all of the low-level needs exactly right before considering higher-level needs? To answer these questions, let’s start by looking at Maslow’s hierarchy.

Notice the author (Steven Bradley) is talking about general design. Ugh! Sure enough, we have some semantic issues here. Damn you language and our plastic language brains.

I feel much of the confusion and debate is sparked by imperfect language and vocabulary. Design, user experience, and usability: These terms are sometimes used inter-changeably. Yet, they can be very different.

Can you use these pyramids to build sites and interfaces? Sure. But, here’s my insight, you CAN test this way – Regardless of if testing Design, User Experience, or usability. First test for the base needs and then test for the higher needs. Why do this? Well, why not. Think of it like a map and a way to segment the complex task into manageable chunks. And, it can help prioritize your testing and help you make decisions and communicate problems to the stakeholders.

For your viewing Pleasure, I’ve collected all the Pyramids on my own private Giza.

Let's get it all on the table

Final Thoughts

Let’s face it – People love charts and pretty bullshit. Pyramids are old and busted – honeycomb is the new hotness. Users are complex.

[Note: In Monday’s podcast I said we were going to be doing a real user test today. We’ve decided to push that back to next week. We had two more items we wanted to get to first.]

When we first started learning about user testing back in August, I was under the presumption that user testing was essentially a controlled, systematic endeavor. But as Steve Krug and Jakob Neilsen will tell you, there’s more than one way to test a cat. (That’s a direct quote, I believe.) A lot can be gained just by getting feedback from somebody. They argue that the name of the game is improving your website (not testing a hypothesis) and that most major issues on a website are visible to most users.

Krug advocates companies set aside one morning a month to do three user tests. Then, debrief over lunch.

This rapid-testing idea was inspiring. It’s like everybody knows what’s not working on your site, and to learn the secret to what’s broken, all you have to do is ask them to tell you about it. Whoa!

You said it.

But is that true?

Are we all walking around with an innate sense of what sucks on websites? And if so, why can’t we design a perfect one out of the gate? If we all have the magical power of user testing, why can’t we user-test our own site and save the time, cost, and hassle of asking other people?

Proximity

The reason you can’t user test your own website is because you’re too close to it. You’re like a director who can’t enjoy his own movie in the same way as a regular viewer because you remember what happened off screen. When you look at your own website you remember the versions of the site that almost were and you know intimately the reasons and compromises that went into creating your current website. You can’t see the forest for the trees.

Your average user is focused on a completely different experience. They are focused on what game theorists call “utility-value maximization”. In plain English, it means that people are goal oriented on websites. It’s built into the way the Internet works. We click on links because we want to access new content. Each click is a statement of purpose. When a user visits your website, they are there for a reason and they have a goal in mind. Han Solo had it right about web design when he said, “Give the wookie what he wants.”

Mental Models

Have you ever helped a relative get a computer or get on the Internet or otherwise interface with a completely new piece of technology for the first time? Do you remember the first time you opened Photoshop or Illustrator or Dreamweaver or the first time you tried to center something with CSS and you had that feeling that you were in completely new territory?

That happens when you have no way to contextualize what you’re seeing with your previous experience.

People create mental models of all their behaviors. Have you ever caught yourself having a dumb conversation with somebody about the weather when (a) you don’t really know why you’re talking to them in the first place and (b) there’s nothing special about the weather? Why do we do that?

Or how about in movies and TV shows when people hang up the phone without saying goodbye first. That’s always weird to me. It’s because my mental model includes goodbye language at the end of phone conversations and in TV shows, it doesn’t advance the plot so it isn’t necessary.

Once you get your non-techie relative online, they too will begin to create a mental model of how the Internet works. Some of it will be based on fact. A lot of it will be based on feelings and intuition that is not correct. (I’m reminded of an old boss who, every time he saw the Blue Scree of Death on his PC would scream “BILL GATES!”) But people don’t get retrained when they learn the Internet wrong. Nope, we just deal and hope they catch up.

Jakob Nielsen points out many of the points of technical confusion in his blog post on the same topic.

Operating-system windows vs. browser windows

A window vs. an application,

Icons vs. applications,

Browser commands vs. native commands in a Web-based app

Local vs. remote info

Different passwords and log-in options (users often log in to other websites as if they were logging in to their email)

In short, when it comes to computers, a lot of us are still getting our act together. What happens as people gain experience using the Internet is they gain an intuition for how websites are supposed to work. And that’s why there’s a great list of usability conventions to use when developing your website.

What does this mean for web sites and user testing?

Signal vs. Noise

All communications come down to the issue of signal vs. noise. Those of us old enough to remember when Sprint was a long distance company who had sound quality so clear you could hear a pin drop have experience with the problem. In any communication medium, whether it’s spoken and heard, or transmitted by radio, TV, print, telegram, fax machine, or Internet, the message must be transmitted and it must be received. The message is encoded in a language and in a medium, and in a context. In order for it to be decoded correctly, the person receiving the message must understand the language, have access to the medium, and know the context in which the message is meant.

See? Easy.

Think of a physical long-distance phone line. The reason Sprint was so happy about their clear signal is because it’s hard to eliminate all the noise. Static, buzzing, clicks, pops – all could (and did) effect the ability to be heard.

When you think about a telephone line, the communication can be broken down into two parts:

The mechanical workings of transmitting a person’s voice from one place to another and,

The content of the communication

If you were to user-test the phone line, you could test for the same two attributes above:

The quality of message transmission and,

The content of the message

Unless you own a tin-foil hat, phone companies today don’t care about the content of your phone calls, texts, and messages. They are focused on delivering on the quality of message transmission. Also, because we all have such a complete mental model on how to make a phone call, it turns out that isn’t the part of the phone business that’s growing now. The user experience for making and receiving phone calls is essentially complete. The new wrinkle is welding the digital side of things to the phone. Now it’s all about experiencing the mobile Internet. And that, invariably, leads to more half-baked mental models for using the Internet.

The Medium vs. the Message

When you add up the fact that there are so many ways to leave a page – from going to another site, clicking the back button, closing the window, getting up and walking away, clicking into a different window, getting distracted, plus the various mental models that people have for how the Internet should work, it’s safe to say that there’s a lot of noise in the channel.

In the physical long-distance phone line we discussed earlier, the medium was the phone line. In your website your critical path is the medium.

Now let’s think about that for a second. Remember how earlier we said that clicking a link is a statement of intent. It could be that the user is expecting to end up on a YouTube video or is expecting to see an Amazon page for digital cameras or is looking for an article on Wikipedia or a house on an MLS. The content is the message. The Internet is the medium. And getting to your content is part of the medium.

We’ve all dropped a call on our mobile phone. There’s a distinct difference between a phone call where you’ve said everything you wanted to and hung up and when the call drops. When the call drops – you can’t communicate any longer. In the same way, if your website can’t get the user to the content they’re looking for, then it can’t deliver the message. It’s a failure of the medium to deliver the message. The noise in your website has to be low enough so that people can find the signal they are searching for.

This signal, the message, is what the message says. It’s the literal content of your website. This is why developing a strong critical path is so crucial. The only way to be heard above the din of distraction is to have a focused message and to provide an obvious way to access that content.

What we’re talking about here is signal quality. And all that’s necessary to test a signal is somebody on the other end to describe what they’re getting.

If we were testing long-distance phone service we’d whip on some glasses, stomp through the woods and ask “Can you hear me now?“. But since we’re testing websites, all that’s required is to ask somebody else what they see on your website.

User Testing in 3 Questions

When you’re looking for quick insights about your website, you’re looking to know something about signal strength. It’s evident in the type of questions that are asked.

None of them concern the content on the website. All of them are testing either signal or noise.

Break down the expected answers:

What frustrated you the most? 99% of the time this is going to be an answer that relates to not being able to find something. It could also have something to do with your messaging.

How would you improve the site? Because you directed them to the site and not what your site sells or provides, this answer is going to relate somehow either to accessing information or clarifying options.

What did you like about the site? This will help with positive feedback and will help you confirm areas of the website that perform well or stand out to the user.

Nothing in there explicitly dealt with the quality of the material. What’s important is strengthening the signal.

I should state that these are not the only questions you could use in a 3-question test. You can test the critical path, a secondary path, or test for comprehension and cognitive load with your site structure without knowing anything about the user.

And we’re going to prove it. Next week, with a little luck, we’ll bring you a video of our own 3 question drive-by user test.

Here are two videos of Ben and Me testing some random websites. We did it to test our screen recording software. And, we did it to get the feel for facilitating a test. And, we did it to test out our New YOUTUBE Channel!

Of course, we had some insights to share – but it’s too dang late for typing this out. We’ll talk about it on Monday’s Better User Experience Podcast!

About a month ago, Newman wrote a post titled 4 Points of Wisdom from Steve Krug’s ‘Rocket Surgery Made Easy’. He was reading the book and wanted to share his insights. At the time, I was immersed in the James Gleick book ‘The Information’. And if you’re a regular reader, you know that led us on a week long journey exploring how entropy is related to web design and user testing. Oh yes, it got serious.

Now it’s my turn to go through ‘Rocket Surgery Made Easy’ and I have 4 more points of wisdom that I learned from reading through the book.

#1. Prove vs. Improve

This was a bit of a revelation to me. When I think of user testing, I think of trying to make a website better. It never occurs to me that I’m trying to prove something. It just seems obvious that I would be trying to improve things. But when it comes to user testing, it’s possible to do both.

Quantitative testing involves creating a testing methodology adhering to a strict testing protocol to ensure non-biased results. If it sounds like a science test, with a hypothesis and all of that, you’d be right. And because there’s a hypothesis and you’re looking for valid data feedback, it is setting out to prove something.

Qualitative testing is much less formal. It’s not focused on proving anything. Instead, its focus is on making things better.

Certainly, there’s room for both types of testing but when it comes to actually doing the testing, for most businesses, it will be more time and cost effective to concentrate on improving things. And doing that is as easy as asking for an opinion.

#2. Why Down-and-Dirty Qualitative Testing Gets Results

Deep down, we all know that nothing is free. So what gives here? Why does it seem like the testing type that requires less rigor, time, effort, and money seem to be the one that actually works? Simply put: it’s because ‘good enough’ is good enough. And by the time you’ve exhausted insights from qualitative testing, you’ll be in a better position to do quantitative testing.

1. All sites have problems

Settle down.

I’ve never come across a website that couldn’t use a little work. Apparently, neither has Steve Krug. One of the main reasons that water cooler user testing works is because there’s always room for improvement.

2. Most of the serious problems tend to be easy to find

In the dust of creating a website, it’s easy to get too close to the whole operation. A common issue comes from troubleshooting problems. These problems can be technical or organizational or even baked into the business plan. Eventually a solution is found and implemented. Sure, it managed to go all Matrix on all of the problems and managed to dodge all of the bullets but that doesn’t mean that the solution was the right one for the user.

The first hint that you're too close.

When you find a non-interested party and get feedback, the major issues will crop up again and again. You can’t see the forest for the trees. They can.

The forest from 'Return to Zork' still haunts my dreams.

3. Getting stakeholders involved in user testing gives them a reality check on who their users are

Another common mistake that’s made during the planning phase of web design is that it’s created for an ‘average user’. The problem is, the average user tends to bear little resemblance to their actual average user. The obvious remedy is to go find some ‘average users’. Technologically, we can do just that.

Use a tool like Inspectlet (which Newman has been using and blogging about) or one of the other ‘Heatmaps / Mouse Tracking Tools’ we have in the sidebar to record your user’s sessions. Share those videos with all the stakeholders and then stand back. Everybody will get a new insight about the ‘average user’ and will immediately want to talk about it. It’s pretty remarkable to watch, actually.

Who you think your users are.

Who your users actually are.

This has the wonderful effect of getting everybody to focus on the right thing: improving the user experience.

#3. Test other people’s websites

This is just brilliant. User testing can (and should!) happen even before the first napkin sketch is drawn. How? Test the websites of your competitors or of somebody else in the same field. Test sites that have features you’re thinking of implementing. A cup of coffee and a conversation could save you weeks of work.

Remember, this is not rocket surgery. It’s basically asking people their thoughts about a website. Nothing says that you only can ask people about your website. As Krug says, “someone has gone to the trouble of building a full-scale working prototype of a design approach to the same problems you’re trying to solve, and then they’ve left it lying around for you to use.”

Websites do two things: provide information for a user to consume and provide a way to filter out all of the other information.

You may know this as our ‘filter’ and ‘confirming’ pages that we talk about again and again. Web pages, until you get to an information page, whether that’s a YouTube video, or a contact page, or a product description page, or MLS search results, are known as ‘filters’. This is because their main role is to get you to the information you’re there to view. The most obvious of these filter pages is the front page.

On most front pages of website, they exist to shuffle visitors off to other pages. They are not a destination page in-and-of themselves. The essence of filtering is clarity. And clarity can be measured by watching how people interact with a page. The easier it is to navigate the page, the lower cognitive load and the higher the success rate.

People want ‘easy’ more than they want ‘better’. If your site isn’t easy, it won’t have a chance to show that it’s ‘better’ because finding another website is just as easy.

This testing can be done even at the napkin-sketch phase. Just ask somebody who isn’t involved with the project to tell you what they see in the sketch. Listen to what they have to say. Chances are they’ll say “oh, this looks like a site for ____ and what’s this ‘experience’ button?” or something to that effect and you’ll immediately hone in on what doesn’t make sense to your users.

Final Thoughts

I can see why this book is the go-to resource for easy and effective user tests. It maintains a laser-like focus on how to improve your website with user testing. It’s filled with good nuggets. Certainly enough to do at least a third installment of this series. I really can’t recommend it highly enough. It’s the perfect compliment for the various user testing tools that we sample here on the site.