Smiley: An app in 24 hours

Late Monday afternoon David, Kiran, and I were discussing how we could begin to measure how our customers felt about our customer service. We’re already measuring things like response time, average tickets per day per person, average tickets in a thread, etc. Those stats are helpful for measuring internal efficiency and speed, but they don’t measure quality from a customer’s perspective.

The idea

We talked about it for a bit and came up with this basic goal: Let’s make it really easy for our customers to quickly rate our customer service every time we talk to them. It’s not rocket science, and it’s not a breakthrough idea, but it wasn’t something we were doing. It was time we experimented with the concept. We’d write some software and try it out. We’d call the app Smiley.

The key

The whole feedback process had to be easy, it had to be fast, and it couldn’t be a burden on our customers. We didn’t want to put people in front of some long-winded complicated survey — no one likes filling those out. We just wanted to ask them one quick question and that was it. The whole thing should take about five seconds and it should be entirely optional. We’d start there and see how it went.

Linked from the email signature

We decided we would add a short link to each support person’s email signature. The link would encode the support person’s ID along with the ticket number for the support request. When someone clicked the link they’d go to our site where they’d be asked to answer one question about the customer service experience they just had. That’s all.

Starting on the design

The next morning I went off and started designing some screens. After a few minutes I had the basic structure. There were five screens total: Three customer facing (and two of those were optional), two internally facing.

(Customer facing) One screen which asked a customer a single question with three possible answers.

(Customer facing – optional) One screen with a single text field where someone could choose to elaborate on their answer. This was entirely optional.

(Customer facing – only seen if someone provides freeform text feedback) One thank you screen someone would see after they submitted their feedback.

(Internal) One screen that showed all our customer service people along with their most recent ratings, their overall average rating, and a link to see all their ratings and feedback.

(Internal) One screen that showed all of someone’s ratings along with any feedback a customer left on a particular rating.

About an hour or so later I had the customer facing screens done. We went back and forth on a few iterations, and experimented with two options (“great” and “not great”) vs. three options (“great”, “fine”, and “not very good” – we picked this version), but overall the design was settled in about an hour. Originally I used some stock photo smiley faces for the mockup, but I asked Jamie to design some custom smileys for the design (you’ll see these below).

The screen the customer sees after clicking a link in the email signature.

The optional screen a customer sees if they answer the first question.

Hooking it up

Next David took the UI and began writing the Rails back-end to make it all work. While David was working on this, I started working on the internal facing admin screens. I spent a few hours messing around with some ideas, but eventually settled on the simplest version:

Internal-facing admin screens

The screen that shows all our customer support people and their most recent ratings. Ratings are displayed from left to right with the most recent top left. This way people can watch trends over time. Note: This is sample data.

Clicking on someone’s name brings to your their full feedback screen. This screen shows all their ratings plus any freeform text feedback.

A few hours later, David hooked up these admin screens. Then we started playing with it all a bit. Walking through it, using it. We realized it would be handy if we could hover over any smiley and see any related feedback. David hooked that up. Then we realized it would be really useful if we could click on any smiley and jump directly to the related ticket in Zendesk. David hooked that up.

Deployed yesterday afternoon

We deployed the app yesterday afternoon, about 24 hours after the initial idea. Support is adding the links to their email signatures and data is starting to come in. We’ll watch it closely over the next few days and see how it goes. We’ll likely explore some tweaks and modifications to encourage more people to give us feedback.

Open questions

Smiley is an experiment. We don’t know if it’s going to work. Will the feedback be useful? Will the feedback be accurate? Will the feedback self-select only great experiences and terrible experiences? Will anyone care? We aren’t sure, but that’s the beauty of just 24 hours from idea to execution — if it doesn’t work nothing was lost in trying. Spending a month or more on something to find out it doesn’t work can be frustrating, but spending 24 hours is just plain fun. It’s a great challenge.

Once we have some data in the system we’ll report back if we discover anything interesting.

Marc Hedlund

I really hope this is going to work, because I hate when customer services ask me to spend fifteen minutes of my time to rate them over dozens of open questions.

Note that your system will help you to rate your employees more than your customer service, contrary to those e-mails and forms I filled which helped companies to accurately rate their customer service process.

I was wondering – how do you cope with clients that express their negativity towards the customer support while they’re just disappointed by the product? Does that skew your stats at all?

Having worked on the FAQ pages of some web services, I’ve noticed many users that downrated perfectly clear answers as soon as they learned that a certain feature didn’t work as they thought it should.

hop

You guys use zendesk eh? I’ve been looking for good, uncomplicated support. What’s the draw of Zendesk?

JF

on 01 Sep 10

I was wondering – how do you cope with clients that express their negativity towards the customer support while they’re just disappointed by the product? Does that skew your stats at all?

We don’t know, we’ll see what happens. I don’t think it will be a major problem. If we see negative feedback we can click the smiley and jump right to the support ticket. We can review the ticket and make a judgement call on the feedback.

Great design. The simplicity, while also great, more importantly fits the need. And I love how quickly you guys went from idea to product because of such a well-defined need.

One suggestion: The “This is entirely optional…” text at the top of the second screen was the very last thing I noticed. This is probably due to both it’s position and the fact that it looks so similar to the previous screen. Maybe look at making that a little more obvious?

JF

on 01 Sep 10

Kyle: I think I’ll probably end up killing that strip at the top anyway. It doesn’t matter. In the end, everything is always optional – if someone doesn’t want to do it they don’t have to do it. People determine their own interest.

zetareticoli

on 01 Sep 10

Nice work!

Too smiley faces thought when a user leaves more and more feedback.

Mike Gowen

on 01 Sep 10

Assuming the design intention for the grid of smileys is to get an bird’s eye view of your performance (i.e. I see a lot fo green!), I’m wondering if there might be a better way to display them. The % is certainly helpful, but it might be difficult to see the balance of smiley types when sorted by recency by default. While not as interesting, maybe a simple 3 bar chart (red, yellow, green) would do it….or even grouping them all into blocks so the viewer could compare the size of the blocks.

Jason: This is excellent. It turns out to be a simplification of the Net Promoter approach to customer feedback (Fred Reichheld wrote a whole book about it, which you can learn about at http://www.theultimatequestion.com.)

To learn more about how companies use this approach, which is VERY powerful, try a blog I keep on the topic: http://www.robmarkey.com.

One suggestion: Don’t you want to ask customers’ permission to follow up with them to learn more about their experience, especially if they were unhappy? Maybe a supervisor or other member of the management team could call all of the unhappy customers, learn more about what happened than they will write in the text box, and even engage in a little service recovery?

One more thought, prompted by Mike’s comment above: I have a PowerPoint template that uses similar “emoticons” we developed for just this purpose. It groups results into a block of one hundred faces, grouped by Promoters, Passives and Detractors (the happy, passive and angry faces, respectively).

The template and the emoticons are available for download from my blog if you register on the site.

JF

on 01 Sep 10

grouping them all into blocks so the viewer could compare the size of the blocks.

We think the actual order is important. Then you can spot trends. This person was getting a lot of reds, but now they’re on a green streak. Or the reverse. Or maybe it’s scattered. That’s useful information.

Of course there are lots of other ways to show trends and group stuff, but this is the simplest most effective way we could do it in 24 hours. Once we have a ton of data in the system we can begin exploring alternatives.

JF

on 01 Sep 10

Don’t you want to ask customers’ permission to follow up with them to learn more about their experience, especially if they were unhappy?

We can. Every bit of feedback is tied right back to a support ticket. Each support ticket is tied to a person and their email address. If we want to follow up we can.

Mark

on 01 Sep 10

Just a suggestion – wouldn’t it be better to have the three options in the email signature? EG “How was your experience with 37 Signals Support? Great, OK, or Not Good?” with Great, OK or Not Good linking straight to the relevant free text entry pages? It would save everybody an extra click.

JF

on 01 Sep 10

Mark: If we could do HTML email signatures we might agree, but three separate full links at the bottom of the email feels like link overload. Requires thinking to parse without the visual cues, etc.

Mark

on 01 Sep 10

@JF: Fair enough. I assumed HTML email. Three big URLs could well be too much.

Three screens? I would put it all on one. Three choices at the top, an “Additional comments (optional)” textbox below it, and a “Thanks for your feedback” blurb at the bottom. Still pretty clean, IMO. Any reason you went with three separate screens?

Lee

on 01 Sep 10

I think it might be useful to also be able to see a history of how each customer has rated support.

You might find some people who are always hard to please. A negative rating from this person might not be as big of a deal.

You might also find a user that was always happy and starts to trend towards the negative. Maybe that is a customer you would want to contact to see if there are reasons why they aren’t as happy with your product anymore.

Rich

Very nice. You could use it internally to rate development’s responsiveness to customer support requests…although I would assume it’s exceptional. 8>)

Thanks for sharing again.

Scott

on 01 Sep 10

Did you use Rails 3?

JD

on 01 Sep 10

Seeing the trends over a period of time will be cool, though using the faces to do it may get kinda noisy. Maybe the binary outcome (win/loss) style of sparklines? In the baseball example, horizontal line connections indicate home/away, but those could also serve as ‘neutral’ feedback. You don’t necessarily need color, but you could do green up-ticks, red down-ticks, and flat gray lines for the neutral. A quick image sample:

Nick Campbell

on 01 Sep 10

I find it interesting that you only thank people for providing freeform text feedback but not for clicking a smiley face. Is there a reason for that?

Maybe you could have it set up with your block set up ahead so that when you hover over green it highlights all the green a bit brighter to make the pattern a little more visually apparent. So you have the mess now, but you want to see how the reds have looked of late for Keiran. Hover over the red and it highlights them and makes them pop from the main block and you can see, hey there have been very few over the past two weeks, but a ton the week prior. What happened there?”

Since people want to see the overall break down you could simply expand the percentage to a green, yellow, red next to the name. In color.

I think this is a great solution, but it has a slight flaw in the usability:

So the user goes to the page and clicks on one of the smileys (they are really happy with the support). Then they get to the page with the text field, and here is the potential problem:
Do they click the “Send this feedback”-button with a blank field to send the smiley-response? Can they just close the window after clicking the smiley and the answer will be saved?
None of these things are clear in the screens shown.

So: The action of sending the feedback is not obvious, potentially making the user feel uncomfortable and uncertain if the feedback is sent or not.
I’m sure it wouldn’t hurt anyone if they submitted a blank field, but the uncertainty is not a good feeling.

But as always great work on the thinking, and kudos for executing as well as you did in only 24 hours.

I love that you did this in 24 hours. It reminds me of the things that get done at startup weekend. Motivated people with a good mix of skills and the goal of creating a working product in a short period of time (visually, functionally, and practically).

Things don’t have to take forever! And I’ll bet this project was a lot of fun to do.

I think the question to ask here is what was your primary goal and bigger win in this exercise, to build a useful and usable customer satisfaction analytic app, or to turn around an app in 24 hours after conception of the idea?

MattS

on 02 Sep 10

Do support people have to add the custom email signature link manually? For every email (assuming most “closure” emails are for separate issues)?

Would be nice to have that automated, if it is, will you share how this is done code-wise?

Anonymous Coward

on 02 Sep 10

Kyle: I think I’ll probably end up killing that strip at the top anyway. It doesn’t matter. In the end, everything is always optional – if someone doesn’t want to do it they don’t have to do it. People determine their own interest.

Why not just “We’re glad you had a great experience! Care to tell us what made it great?” It still a question, but offhand enough to not sound forced.

Tim W.

on 02 Sep 10

Well, I think the idea is nice but it’s unlike you guys to make things more complicated than they actually should be.

Why would you want to now if service was great? Or even OK? What would you do with that information? Because not every “happy” or “so so happy” customer would click on the link, let alone the smileys, that information would likely be statistically irrelevant.

Seems to me you could change the email signature to “Not happy with our service? Click here to let us know” and reduce the number of customer facing screens to just ONE: “Thank you for being so kind to let us know you are NOT satisfied with our service. If you care to elaborate on what happened, please fill out the below comment form and we’ll get back to you as soon as possible”.

tom

on 02 Sep 10

same thing in Statoil gas stations (Europe) for over 5 years now.
implemented as a small terminal which you click.

I like this because I think it invites the customer to think about the support they’ve received — to think about it and have a conscious opinion on the service.

If it’s negative, they’ll probably dig deeper. If it’s positive, they’re more likely to remember the experience. Usually, we only remember negative experiences. No one ever says “wow, the air conditioner temperature was perfect in that room”.

Also, I dig the choice of positive/neutral/negative — it would have been tempting for many to have a huge sliding scale of emotions (just like surveys usually do), a 5-star rating (like movies and iTunes) or some other complicated system.

Interesting application. I came across recently with a company that provides a similar service http://nicereply.com/.
They present the stats with graphs instead of smilies.

I agree, too many smilies could become a little bit confusing, I would replace them with 3 simple progressing bars, red, green and yellow.

JF

on 02 Sep 10

Why would you want to now if service was great? Or even OK? What would you do with that information? Because not every “happy” or “so so happy” customer would click on the link, let alone the smileys, that information would likely be statistically irrelevant… Seems to me you could change the email signature to “Not happy with our service? Click here to let us know” and reduce the number of customer facing screens to just ONE : “Thank you for being so kind to let us know you are NOT satisfied with our service. If you care to elaborate on what happened, please fill out the below comment form and we’ll get back to you as soon as possible”.

Because only getting negative feedback isn’t good for the soul. If the only customer feedback you get is negative then you’re naturally not going to like customer feedback. It’s uplifting when someone tells you are doing a good job. It makes you feel better and when you feel better you do better work. And then when you do see the occasional negative feedback you’ll pay more attention to it and put more effort into learning from that experience.

Matt

on 02 Sep 10

@JF: Totally, Rad? [/pun]

I dig the streak approach. Maybe a toggle “Streak” and “Block”.

Block could group the greens, yellows, reds…visually it would make it like a big fat bar graph.

And, the “Streak” view (which is funnier every time I type it) would show you how things naturally played out.

It’s quite a coincidence that just a few days after Zendesk let every customer including 37signals know about its integration with Nicereply.com (blogpost: http://www.zendesk.com/blog/get-feedback-on-support-responses-with-nicereply) , you came out with Smiley claiming it your original idea, even if the functionality and idea is exactly the same as ours ( even the zendesk integration )

I understand that odds are you came up with this all by yourself, but with this timing it’s just hard to believe. I would be very disappointed if the author of our favorite books about business practice would just blatantly copy our idea. I understand it’s not rocket science but we haven’t been able to find a single service doing this kind of email feedback.

Don’t take this the wrong way, we’ve always been fans of 37signals, and that’s why we’d like to know if this was honestly just a coincidence. Anyway, hello to our first competition!

DHH

on 02 Sep 10

Hi Matej,

The first we learned of Nicereply was in comments to this thread. Congrats on launching a business, though. Even though they are similar, Smiley is obviously a lot simpler because we didn’t need much and we don’t have any current plans to turn it into a product.

DHH

on 02 Sep 10

Matej, also, asking your customers for feedback at the end of an email or phone call is hardly a novel idea. Not that it needs to be, just that no deep dish was invented here. Most surveys then end up being pretty complicated, which both Smiley with just 3 ratings and Nicereply with 10 make a lot simpler. So there’s definitely a philosophical allegiance to wanting to take a tried-and-true model and simplifying it. Which is great.

Scott

on 02 Sep 10

In support of Matej, I can see how his curiosity was piqued by the coincidentally of the announcement and your efforts. Zendesk profiles Nicereply on Friday. On Monday you come up with the same idea. If I was Nicereply, I’d probably post the same question. DHH, Thanks for responding promptly to Matej’s question and clarifying the coincidence.

DHH

on 02 Sep 10

Scott, fair question. In this case, it was indeed just a coincidence.

But to be honest, even if we had known about Nicereply, there’s a fair chance that we’d still have done Smiley. I don’t personally like the 1-10 rating system, so since we wanted something simpler than and since we could build it in a day, we’d probably still have done that.

Sorry, but just because an app already exists doesn’t mean you can’t create something similar yourself. This is done everywhere in all types of markets.

Throwing a fit because someone made something similar to you is childish. Plus, you weren’t the first person to come up with customer satisfaction surveys to begin with, which means you are just as “guilty” as 37signals.

The only thing you both are guilty of is taking something cumbersome and lame (like annoying customer surveys) and improving on it.

I think it would be useful to give some indication of users who didn’t respond/pick a face. Right now there’s no way to really gauge what the responses reflect as a % of the person’s total tickets. And not selecting an option is an implied neutral response which should be represented. (I’d argue your middle choice here will actually skew to a slightly negative tone – they’re likely people who were dissapointed but not upset.)

I’m not sure what response ratios you’re getting so far so hard to make suggestions (as the volume of non-responses would affect how you would display the info).

If there was a good split then you could consider displaying a gray, neutral face but if you were only getting a 10% response rate or something that would blow out the coloured faces – in that case you could just display the total # of tickets (i.e. 75 responses | 200 total tickets) .

Dima

on 03 Sep 10

Could not read through all of posts, so sorry if it already was there.
Good solution, but I fail to see the logic of client going to some link first of all and evaluating support there while he can basically do the same stuff right from his email.

Imagine: no odd clicks – just three happy faces right on bottom of an email message and ‘Rate’ button (of course, the opportunity to text feedback should be provided).

I believe that it is more elegant and simple solution.

BR,
Dima

JF

on 03 Sep 10

Imagine: no odd clicks – just three happy faces right on bottom of an email message and ‘Rate’ button (of course, the opportunity to text feedback should be provided).

We imagine it too, but we can’t do it because we can’t use HTML in the support emails.

However, yesterday we started experimenting with three direct links at the bottom of the emails. It’s a considerably longer footer, but they are resulting in more ratings so that’s a definite positive.

Scott

on 03 Sep 10

It’ seems that the Internal-facing admin screens are very busy with all smile faces and would get increasingly so with time. It, in my mind, would be easier to read if it were just three Icons ie..

Sarah:

:-) 54%

:-| 32%

:-( 14%

Kiran:

:-) 50%

:-| 30%

:-( 20%

Functionality could be added to the smile faces in that by clicking on them it would show all the comments for that person in that category like so.

Sarah:

:-) 54%

Comment…..........

Comment…..........

Comment…..........

Comment…..........

:-| 32%

:-( 14%

This would eliminate the need for the person’s name on first Internal-facing page to be a link to the second while also eliminating the need for the second Internal-facing page.

Jay Schumacher

on 03 Sep 10

@Scott – If you take that approach you lose the trending, which I believe both Jason & David have mentioned as being the primary reason for displaying the Smiley faces in that manner.

Scott

on 03 Sep 10

@Jay,

I thought about that as well but my “Post” was a little premature. I was trying to get the formatting right and well… We monkeys are trained to hit the button for a reward. :-)

I see using the smile faces, as presented, for a trending indicator to be problematic. The number of faces at any given period could grow to be huge or when feedback is split such that it’s difficult to tell what’s going on.

The faces are just not concise enough for my taste. I found that I had to look at them longer than I would like in order to understand what was going on.

My preference would be for something akin to a stock ticker.

For the group as a whole it would show the faces, the percentages and the trend over the past week, month and year at the top of the page.

For each individual; Next to each smile face percentage it would show the % change for say the past week or month.

Kris Lynch

on 03 Sep 10

As a user that would click the optional link to see the survey, I would also then be happy to answer the one question given the simple clean design… However, I would want to know that I could elaborate on my answer before choosing…. Often when I have a mixed experience and there is no obvious way to express this (loved the Agent’s attentiveness but think that the service was poor because the Agent couldn’t help me) I will simply skip the survey. Therefore I suggest that the design could only benefit from some single line on the first screen noting that there will be opportunity to elborate on your experience if you choose. Either that or simply insert the free-form box visibily on the same page and activate it only once a choice is made. I do really like the simplicity and clarity expressed in your presentation.

Jay Schumacher

on 03 Sep 10

@Scott – Sure, I see what you mean. Personally, I kind of like what they did and I like that it’s less “formally” presented. If the audience on the internal side was wider, or the list of people longer, your approach would probably make a lot more sense. All in all, for a quick turn-around, low investment product, I like how it’s done. Once they prove that they are getting good data behind it, putting more time into more efficient presentations would make sense in my view.

What would be interesting would be to see some type of feedback for the blog, since this is the primary medium you use for sending you message out to the masses. I think I see you trying to do this with the twitter button, but I don’t believe people retweet things that they like, believe in or support necessarily. God forbid you put a FB “like” button as well. Just some mechanism that let’s you know if your messaging is hitting the audience in the right way.

The idea is a lot of fun, and I look forward to hearing about the results. I am curious if changing the smilies to have solid colors instead of gradients would help the trend visibility in the aggregate views?