I Love You, Spotify … Now Change : A web app review in the form of a guerrilla usability test

By running Spotify through a usability test, one avid user was able to find some relatively easy fixes that could make for a more intuitive and rewarding experience.

Article No :1282 | July 30, 2014 | by Lin Wang

I’m a big indie music fan and Spotify provides my daily soundtrack. I use it happily all the time, but, as with any great product, there’s still room for improvement. As a quick usability test revealed, people run into critical issues using some Spotify features.

My Objective

Identify the pain points of finding, organizing, and sharing music within the Spotify web application on the desktop.

Who: Eight existing Spotify users who listen to music everyday, though not necessarily using Spotify

Where: San Francisco

Participant Selection

I created a persona before I conducted my user tests and used the persona to screen my participants. For example, I posted on Craigslist with the criteria listed below. It helped me filter for the right participants. Meet my persona, Nick!

Nick is an iOS developer who loves music

Test Tasks

Create a personal playlist

Add song(s) to the playlist

Edit the playlist and share it

Subscribe to (follow) artists

I determined these tasks based on the essential needs of using online music streaming services. I phrased the tasks as open-ended scenarios to avoid leading the participants to complete any task in a predetermined way.

Processing

The eight usability tests were recorded using QuickTime. I reviewed the recordings, took notes, and identified and prioritized usability issues.

Notes taken at each session with major pain points reflected from users

Pain points bucketed according to user actions

As you can see, there are a lot of issues going on here and I am not going to address them all here. Instead, let’s take a look at some of the focused changes that could make using Spotify a more enjoyable experience.

Issue 1: Search Results are Hard to Filter

Users expect to be able to customize their search results. As one of my users said, “The reason I don’t use Spotify is that it is so hard for me to find anything.”

The picture below demonstrates that the user wanted to find a song that contained “tourist” in the title. The user typed “tourist” into the search bar, but the results were impossible to sort or filter. The user could not find the intended song.

How about we try this?

Allow users to sort/filter song lists by artist or album or even date of release?

In the case of Spotify’s filtering system (shown below), there are filters—such as “Albums,” “Artists,” “Profiles,” “Playlists,” and “Songs”—but the way Spotify lays them out confuses users: they are visually too similar.

How about we try this?

There is an opportunity to subtly differentiate these kinds of results that I’ve detailed below.

Issue 2: The Tooltips Are Confusing

Users expect the “+” to mean “add to,” instead of “save,” “follow,” or “blank,” and Spotify’s “+” was confusing to my users:

“By clicking on it, I thought it would add the song to my playlist. Where does it save to?”

“Gosh, this is confusing!”

“Why is there such a function?”

Every participant clicked the “+” sign to add their first song to their playlist. Half of the participants still accidentally clicked on it even after they realized that it was wrong. This icon also appears to be “follow” and “null” on different pages, as shown below. The inconsistency confused the participants.

When moused over, the “+” says “save” but no one knew what this meant or where the song was saved to (As of May 21st, 2014)

Here, however, “+” means “Follow,” which is confusing because it previously meant “save”

On the playlist page, this button has no tool tip, which is obviously not helpful

How about we try this?

Give the “+” icon an “add to” function, which enables users to choose which playlist they want to “save” the song to.

Issue 3: Why Am I Following?

The “Follow” button confused all the participants. Each one thought the artists they followed should be listed under the “Follow” button on the left panel, but they are not.

“Where are my followed artists?”

“What does ‘following’ mean?”

“I’d still just use Search to find the artist instead of using this ‘following’ thing.”

The participants expected the “Follow” page to show them a list of the artists they are already following. Instead, the page has only the three options illustrated below.

The Follow button on the left panel actually takes users to a page to do the action “follow”

How about we try this?

Add the “followed artists” section into the “Follow” page. Show news feed, tour dates, and merchandise information as a clear option. A design recommendation is shown be-low.

There’s no doubt Spotify’s current design is clean, but is it informative enough? At this point, could replacing icons with named features benefit the UX without damaging the visual elegance? As Burka reminds us: “Having to hover over a button to deduce its purpose is poor UI even if it’s super compact.”

I did additional user interviews around this concept and found a few problematic organizational problems in the navigation panel.

The original Spotify navigation panel

I also think the big buttons are more appropriate for the touch-screen devices, and a more traditional desktop application (e.g., iTunes) approach would make it easier for users to understand on desktops.

So in an attempt to bring more information to the surface of the interface , I made some design suggestions as shown below:

Conclusion

The goal of user research is to understand the target user’s needs and motivations so that we can develop products that they truly love. By running one of my favorite apps through a usability test, I was able to find some relatively easy fixes that could make Spotify even more rewarding for users. (The suggestions still need more validation and testing).

Do you have any suggestions for changes that could improve the Spotify experience? Have you ever run a usability test on one of your favorite products? I’d love to read your thoughts in the comments below.

About the Author(s)

Lin Wang is a full-stack designer and front-end development enthusiast who is passionate about technology. She loves creating user expereinces that make people's lives more enjoyable. She lives in San Francisco.

Comments

Thanks for this write up Lin. It was very well structured and written and as a new UX designer I learned how to translate my everyday frustrations with interfaces into test prototypes to add some heft to my intuitive issues. I learned a lot but two very important insights were gleaned from this article:

Users are really patient with sucky design. We tolerate a lot of cumbersome stuff.

I use Spotify everyday as I love continuous music stream of my own making rather than listing to the radio which mixes genres and creates a horrible listenting experience. And each day I get frustrated by it's total lack of attention to affordance and conventions and consistency. The multiple functions of the "+" is a prime example of that. I thought it meant "add," but it can mean a "save," or "like," or "follow," and a couple of other things I haven't stumbled upon yet. So just to test something I went back to Spotify and I just wanted to simply add a song from my radio station to a playlist. And there was that dreaded "+" sign again. I clicked on it expecting it to add to a playlist because the link immediately above it says "New Playlist +" but of course it just gives me a checkmark and I'm left with no feedback. What the heck did I just do? Well, what it does is just add to something generic called "Your Music.."Now if you hover over the song playing and not look at the song listing on the left it gives you three symbols "+" "audio icon" and "..." more button. So I click on the "+" which to me still means "add," and yep get the check mark and no feed back. What did I just do? You have to select "..." get the drop down menu and select the option to "Add to Playlist.." I'm tired just typing this.

:)

Thanks for the write up. It was excellent. And I think a lot of the reasons people aren't paying for Spotify is that it's interface is just to cumbersome. Why pay to be frustrated when I can tolerate crap for free?

Well done Lin, it's great as UXers to take a critical look at the designs we use everyday. Like you and the others, I love my music. I started with Pandora then migrated over to Spotify like many others. :)

I think you have identified many of the issues. I would like to add that the IA of the site is quite challanged. In it's attempt to do everything for everybody, the organization of menu items, etc... seem compromised and results in a bit of a convoluted arrangment. E.g. I'm at odds with the Main selection panel on the left side... items like "Play Queue" and "Devices" seem like they should be better integrated into the design, versus as a part of the "Main" selection list.

I think another challange for users is the lack of a "common fate" aspect in the design. What I mean is the doing/selecting/etc of something and the resulting information updating the main sections of the design relative to what was selected.

Where Spotify is at odds with this design technique is when a user has his playlist going, then selects another item in the app which visually updates the main panels, while the playlist is still playing behind the scenes... kind of a visual dis-connect for me. I think I would prefer a more "grounded" experience, where my playlist queue or whatever is being played (Radio, etc...) has a constant visual presence no matter what else I am doing in the app. Lack of this kind of visual connection I believe results in some confusion for new and casual users.

Agree on all counts, but it's missing a couple things, specifically in the area of IA. Spotify's metadata is wretched. It cannot differentiate between two artists of the same name, nor can it tell if a song is a cover version of another or a title with the same name. Occasionally this has led to discovering new music, but more often it leads to me finding crappy rap or pop songs with the same name as an awesome song. Before any UI cosmetics, Spotify needs to address gaping holes in IA.

I wanted to add that the task of adding metadata is certainly not insurmountable. Sound recognition is advanced enough that you could match melodies automatically to sort songs into cover versions, at least 99% of the time (there are some awful covers out there, I know).

This is an excellent article. One I can sink my teeth into because I too am also a lover of music, and I am a user of Spotify. The company I work for is really considering many elements of UX in our design process. As an application designer I can appreciate adopting a user mindset in my design. One thing I did notice about the article is that you must've missed the QA phase of publishing the article. One of your images led to a "Page Not Found" The image in question is the second one under "How about we try this" in the "Issue 1: Search Results are Hard to Filter" section.

I know I am probably being a bit of a nick-picker, so do with what you will.

It's a great web app, even tho to get the full experience, the spotify app is available on many platforms. I see this web app to be more of a competitor to Grooveshark then anything. Well, I can use the web app/site of grooveshark on my phone, I can't on spotify web app.

Thanks Lin for your great article. Maybe its even better to rethink the semantics of the plus sign as a representation for the "more-options" dropdown to help the user to differentiate better between "adding" and "more". A better representation for a dropdown would be three-dots, three stacked lines (burger menu) or whats learned for desktop, the right click for bringing up the options.

Great article, but I am SO disappointed you didn't tackle the play/queue confusion, which is what makes me want to chuck spotify out the window every time I use it. Seriously. Click Queue on on album. Click Play on another album. What did you expect it to do? Did it do what one would expect it to do? I bet not ;-)

Great article Lin. Someone is finally running a usability test on Spotify! ;)

I've been thinking about writing an article about the Spotify problems as well, only from my own point of view and about the mobile/OSX apps. Quite a few problems you've highlighted I personally ran into as well.

Like you, I do like Spotify, and I use it daily. Unfortunately the UX side of things seems to have taken a backseat to the visual design when they redesigned all the apps a while back.

The apps used to get out of my way and let me do what I wanted....listen to music. Now I constantly have to be aware of everything and really pay attention. Consistency is lacking, they don't use shapes/proximity/colors enough to make things clear. Wrong type of controls, not matching the mental models of people.

Replacing the yellow star with the grey + sign and checkmark was one of the worst changes imho. Everybody understands what a star means, it is used in almost all apps these days. In the OSX app it doesn't even show a tooltip and you have no clue where the song is going to end up when you press the + sign. No feedback at all. And for the life of me I can't find a reason that change would benefit the user in anyway.

Very good, Lin. I really hate trying to find who I am already following. Also, it would be nice if you could create playlists that you could grant access to friends that you chose, so you can all contribute to the playlist.

You should've seen their Windows app. So awful to use. Confusing, cluttered and barely readable. I wish they would pay more attention to the users. The content and features seem to be great though but they are barely discoverable.

Why show the user results that don't match their intent at all? If the user wants to find a song with "tourist" in the title, then allow them to specify Title as the search and eliminate the screen clutter

That would seem like the logical thing to do if people thought about all their actions before they started doing them. People behave very differently in reality though. Instead of figuring things out upfront we try the first thing that seems to make sense. So with a search box we start typing immediately. Returning multiple categories works much better, at least if you can easily tell them apart. You don't want a user to think about how to search. They don't care, as long as you produce the results they are looking for.

I would agree if it were something like Amazon where the search can be for just about anything. But I think we're pretty focused here. It would be hard to imagine a situation where the user wasn't sure if a word was part of a song title or part of the artists name. Certainly give the user the flexibility about how they want to search, but I always opt on the side of progressive disclosure.

I understand your reasoning but I look at it the other way around. If it's so focused, why bother the user with it at all? What's the chance of them not finding what they are looking for?

If the list of results is too large they need to refine their search (or filter them) anyway, no matter if it's a song, album or artist. And the best way is often to refine the search by mentioning both the title and the artist. Unfortunately if I search Spotify for the song Next Breath by Tank, "next breath tank" works but "next breath by tank" doesn't.

They should also have enough data to know which type of searches are the most common and order them accordingly. For a lot of people search is like google, just type anything you want in there.

Yes, a person will know if it's a title or artist if you ask them. But even if you'd ask someone, they'd have to think about it, even if it's not very long. And what if they want to combine them in the search.

Even if your app is focused, the user often isn't. If they don't have to make a decision, take action or have to think about what they are asking, the happier they usually are with your product.

Is it always logical? Not at all. People almost never do what's the most efficient but what's the easiest.