Finding and Interviewing UX Superstars

In our previous article we talked about how to define the user experience role you need to fill in your organization and how to create a job description that will appeal to potential candidates. As we noted, hiring user experience professionals can be a massively time-consuming endeavor that involves navigating a mysterious and hyper-competitive talent market. Thus we are trying to take some of the confusion out of the hiring process.

So, now that the position has been defined, it’s time to review and evaluate candidates and locate the best match for the job. There are a few steps to this process, and they pretty much mirror the steps followed in any hiring situation.

Create a pool of potential candidates to fill your role

Set up an interview panel

Develop a set of interview questions

Conduct the interviews and assess the candidate

Create a Pool of Candidates

To start your process you need to find people who might be a possible match for your job. You’re likely to get a number of applications in response to your job posting, but you shouldn’t limit your search to just these. Be proactive and look for people who you would want on your team. Fortunately, there are a lot of tools at your disposal to find them.

Ask your teammates for internal referrals.

Contact universities that have programs in design, human-computer interaction, and similar disciplines.

Reach out to UX groups on professional social networking sites, such as LinkedIn.

Look for resumes and portfolios of people who work in companies that build products you consider to be well-designed.

Resume review

You can learn a lot about UX professionals before you ever speak to them by looking at their resumes and portfolios, which will help determine the candidates who are worth investigating further.

Ideally, these resumes will show the impact of their UX work on products that have already launched. Even new graduates should mention some practical industry experience, either through internships or school projects that involved real clients. You may choose to favor candidates who have experience with companies or domains that are similar to yours.

The quality of a resume itself is also important. Is it easy-to-follow, short, and succinct? Did the candidate proofread it? Is the layout well-designed and consistent? After all, you are looking to hire someone who will be involved in designing your product. If they can’t create a forceful and attractive resume, consider that a red flag!

Portfolio review

The vast majority of UX professionals, with the exception of some researchers, will have an online portfolio to showcase their work. Review these portfolios carefully and you’ll gain insight into the breadth and the quality of your candidates’ work.

Take a look at the projects presented. Are the final results good? This may feel very subjective, but it’s important to know that the person you hire is going to satisfy your company’s requirements. How many projects are presented? Do they show a wealth of skills and demonstrate the ability to work within a variety of domains on different UX problems?

In addition to the examples of the completed work product, a good portfolio will detail some of the processes that led to the final results. Ideally, for design projects you’ll see the project brief outlining initial requirements, sketches, or wireframes that led to final design decisions, and a note on timelines. If a portfolio only shows final screenshots it's much harder to understand the contributions of each person. Remember, UXers never work alone! You need to know whether your candidate was a key player in the project or had only tangential involvement.

Understanding the real-world constraints of the projects is important for your ability to assess the outcomes. School projects will often have a lot of flexibility, fairly large teams, and semester-long timelines. Out in the field, UX professionals don’t always have the luxury of being able to follow the ideal design or research process due to time or budget constraints. Real-world projects have unexpected twists or requirements that can affect the finished product in many ways—UXers need to be able to adapt and still produce high-quality work.

The portfolio itself is one of the pieces that the UXer has created. Does the designer understand how to use type, color, and space to create an online presence that is elegant and usable? Does the researcher deliver findings in a way that’s engaging to an audience not familiar with the project? Is the site easy to navigate? Are there any obvious red flags, such as technical glitches or annoying Flash intros? Don’t hesitate to view the source code of the site to check the quality of HTML, CSS, and JavaScript if you are looking for someone with solid technical skills.

You may not be able to get all the answers you need from the portfolio, and if you are interested in proceeding with a particular candidate, you’ll have a chance to probe more deeply into the details of these projects during the interviews. Once you’ve found a manageable number of candidates to move forward with, it’s time to set up your interview panel.

Set up an Interview Panel

As you prepare to speak to the candidates, you have two crucial tasks. First, you need to figure out who in your company is going to be on the interview panel, helping you assess the UXers. Second, you need to educate yourself and the rest of the interviewers on what questions to ask.

Ideally, you should have five or six people participating in the interviews. One person will conduct the phone screen and will join the remaining interviewers for the candidate’s on-site presentation. Aim for a diversity of skills on your interview panel. If you already have UX professionals in your company, they should certainly be on the panel as they are best equipped to evaluate your candidates’ core skills. Additionally, enlist the help of one or more of the key decision-makers, like product managers or engineering leads. They’ll be working with UX closely, and it’s important that they have a say in who you hire.

At Google, we’ve found that we get the best results from the panels that are somewhat skewed towards the candidate’s specialty, but also have input from cross-functional stakeholders. For instance, interaction design candidates speak with two or three designers, a technical interviewer (such as a frontend developer or a prototyper), a researcher, and a product manager or an executive. Similarly, research candidates speak with a few researchers, a designer, and a PM. Don’t fret if you are just building your UX organization and can’t yet set up an ideal panel. Read on to learn about what questions to ask.

Develop a Set of Interview Questions

If most of the people on the interview panel have never interviewed UXers before, you should invest some time discussing questions to ask and possibly even assign areas for each interviewer to cover. You want to avoid duplicate questions and ensure that you can consistently compare interview feedback across candidates.

In addition to role-specific skills and user advocacy aptitude there are attributes that all UXers should have: communication and collaboration skills, a desire to have impact on real-world products, abstract problem-solving prowess, and the ability to prioritize projects.

Role-related core competencies

You’ll be able to assess the level of production skills for designers from their portfolio and design exercise, but you should also focus on their design process and project management skills. Can they work across functions to gather insights, start from sketches, iterate on ideas, and eliminate bad ones, wireframe potential solutions, and collaborate with engineering to get things built? They should be able to initiate and lead design projects to completion while incorporating constructive feedback into their work.

User researchers know how to ask the right questions and refine those into a specific research plan. They need to have a good handle of several methodologies and be able to analyze data effectively and without bias. Experienced researchers will adapt common research methods to the situations and questions at hand and discover answers within reasonable timelines. They may also rely on synthesizing data from past research and external sources. They’ll take initiative and propose research if they feel that the team can benefit from learning more about a particular aspect of user behavior. Finally, they know how to conduct ethical research.

In addition to good knowledge of design and development processes, prototypers should possess a strong technical understanding of front-end coding. They should be able to create prototypes to express design ideas that the rest of the team can relate to. Can they be the bridge between UX designers and the engineering team? Prototypers need to utilize technical skills at their disposal—HTML, CSS, JavaScript, Flash—to bring interaction ideas to life.

Communication and collaboration skills

You want to find out whether your potential hire is able to communicate clearly, confidently, and concisely both orally and in writing. UXers need to know how to fine-tune the delivery of their design proposals or research insights to the right audience, including those of non-UXers. To succeed in their new role, an ability to build alliances across the organization is absolutely crucial.

These soft skills are tough to test for. Your best bet is to pay close attention to the onsite presentation and ask very detailed questions about the portfolio projects. Ask about difficult stakeholders; probe on the team composition and the outcome of their design or research projects; inquire about working with teams that may have not had prior experience with UX. The best candidates will be able to base their answers on past experiences and will show a great degree of self-awareness, discussing team engagement strategies that have worked for them before.

Desire to have a positive impact

Consider diving into your candidate’s previous projects to find out their level of experience working in the industry. This especially applies to new grads and researchers coming from academia or research labs.

The UX field is exciting, and designing clever new interactions or tracking down answers to fascinating research problems can be very engaging. But what are the outcomes? If your aim is to build a business, you need UXers who care deeply about the impact they will have on your products and customers.

Do they focus on users throughout their design and research process? Do they know how to provide actionable recommendations? Can they be productive within your budgets or development timelines? Has their work led to product improvements? Do they know how to get development teams to buy into their suggestions? How do they feel about working on someone else’s ideas as a part of a bigger team? Can they work around technical constraints and adjust their recommendations accordingly? Listen carefully since these answers will help you identify the folks who have the ability to make your products better.

Abstract problem-solving

Problem-solving is not an easy skill to test for but it’s crucial for UX professionals. Try throwing an “unsolvable” problem at the candidate and observe how they try to crack it. As unfair as it may sound, it’s a very useful exercise. Think of a design problem for which there is a general approach that makes sense and have the candidate talk through their plan of attack. Encourage them to sketch their ideas if it helps to explain the concepts and be careful not to impose your own solutions.

As the conversation progresses, feel free to provide feedback on the ideas and throw in additional constraints. It’s important to see if the candidate is able to adapt to the changing requirements, incorporate your feedback into their thinking, react to roadblocks, and work with others.

Ability to prioritize UX projects

Just like with most other jobs, there is always more UX work to do than one can handle. The dichotomy of urgent vs. important projects is not lost on designers and researchers, and the best UXers are able to balance the short-term pressures of tactical work with long-term investments in strategic engagements. If you are looking to hire one of the first UXers in your company, this person will likely support multiple features, products, or teams at the same time. The hire needs to be able to gauge the potential impact of UX involvement and strive to take on the high-impact projects, prioritizing UX issues from “nice-to-fix” to “launch-blocking.”

This is where situational questions and case studies come in handy. You can ask your candidate to come up with a proposal for a design or a research project on a particular subject. Don’t expect anyone to craft an ideal solution during the interview. What you are after is the questions they ask you! Did the candidate want to learn more about the current priorities or try to clarify the team’s needs? Did you have to provide additional information on what the team already has in place, the goals of the project, and the timelines? All these questions are something UXers should be asking when they engage with a real team on a real project, and it’s fair to expect the same approach during the interview.

Some final thoughts on the interview questions

As you can see, many of the questions you could ask involve case studies and design or research exercises. A good rule of thumb, however, is to avoid asking questions that are specific to your business. Don’t ask the candidate to share ideas on redesigning a feature you are struggling with! The candidate will always be at a disadvantage since you have a lot more information and context about the problem, and you’ll rarely be satisfied with the response. You also run a risk of getting into some intellectual property hot water if you choose to implement a suggestion from a candidate you don’t hire. Stick to neutral questions about sites, products, or experiences and let your imagination flow! We’ve asked candidates to redesign or study grocery stores, public transit stations, ticket sales machines, elevators for 1000-floor buildings, and more—fun challenges that give your candidates ample opportunity to show their user-centered thinking and creativity.

Conduct the Interviews

Phone screen and design exercise

Interviewing candidates on the phone is a fast and cheap way to get a feel for their skills, experience, and culture fit. It’s also a good opportunity to tell them more about the role. Phone interviews work best for questions that are easily answered without any reference to visuals. Probe on a candidate’s design process, teamwork, and past experiences—this is your chance to figure out the role they played in the projects presented in their portfolio.

You may ask your candidates to complete a design exercise after the phone screen and get a “live test” of their design skills. The formula for useful design exercises is quite simple. You want to give all your candidates the same prompt (or a choice of two or three prompts) a few days before the onsite interview. In the interest of fairness, limit the time they spend on the exercise—three hours is usually enough. Your goal is to understand their design thinking, ability to focus on the user, and level of expertise across interaction design, visual design, and technical skills. Outline the deliverables you are looking for in your prompt: description of the process, intermediate work product (e.g. sketches), wireframes for a few interactions, and one or two high-fidelity mocks or an interactive prototype, depending on the role you are hiring for.

Design exercises are also a great way for your interviewers to calibrate themselves—over time they will see multiple candidates present solutions to the same problem.

Onsite interviews

This is the most exciting part of the hiring process (maybe aside from getting your offer accepted)! Let the candidate know in advance that interview will start with a presentation where they talk about their work in front of the interview panel. The presentation should take about 45 minutes to an hour and include a portfolio review and a design exercise, if applicable. It’s a way for you to gauge the candidate’s communication skills and design or research process.

Individual interviews are the final step in assessing your candidate’s fit for the role. Each should last about 45 minutes to allow every interviewer to follow up on all the questions they may have from the portfolio and design exercise presentation as well as ask any additional questions. This is also your chance to get into questions that can’t be asked in other settings, such as design or research case-studies, abstract problem-solving, leadership skills, and so on.

Assess the candidate’s fit for your role

Phew! You are almost done! Your team had a chance to speak with the candidate, and the last step is to decide whether you want this person to join your company. Ask everyone to write down their feedback and schedule a team discussion as soon as possible after the interviews.

Listen to everyone’s thoughts, as your teammates will be bringing diverse perspectives both on the candidate’s skills and the culture fit. If the feedback is uniformly positive, you are in luck. If there are some concerns, discuss whether or not lack of certain skills is a deal-breaker for your position. In any case, don’t let anyone dominate the conversation!

The most difficult decisions are usually the ones when everyone liked the candidate but nobody was blown away. Is “competent” good enough for your team or would you rather search for a superstar? Our advice: hire people who are better than you at what they do and who bring new, unique skills to your team.

About the Author(s)

Olga Khroustaleva leads the User Experience Research group at YouTube. Her team is tasked with improving the product experience for 800 million unique users who visit YouTube every month as well as YouTube's content providers and advertisers.

Tom Broxton is a UX lead for Monetization and Devices at YouTube. Recent work includes AdWords for video, the self serve tool for video advertisers; TrueView skippable video ads; Insight, YouTube’s first generation video analytics tool; and research into the spread of viral videos. Prior to YouTube, Tom worked as a designer, consultant and creative lead in the UK with organisations such as Brandwatch, UpMyStreet.com and BBC News Online.

Comments

There are a number of problems with the recommendations in this article, which others have pointed out. However, one thing I'd like to highlight is the statement:

"Interviewing candidates on the phone is a fast and cheap way to get a feel for their skills, experience, and culture fit."

I would respectfully suggest that this is utter tripe. Phone interviews are not a good idea at all unless you are forced to do them for reasons of distance. Even then, attempting to judge crucial things like team fit or even skills is next to impossible. I'm surprised somebody working in UX would even consider that using a phone interview would be on a par with interacting with somebody face to face. Personally, I only call prospective candidates if there is a specify thing I need to clear up before deciding to bring them in for an interview. Just 5-10mins maximum. If an employer wanted to give me a full interview over the phone, I would mostly likely refuse and demand a face-to-face.

As regards the final statement "Our advice: hire people who are better than you..." On the strength of this article, I'd say the authors would have no problem achieving that!

I was about to read the article but saw that the writers are working at Youtube, a web site with one of the worst UX. Now I am going to read the article and do exactly the opposite of what they are saying.

I’m astonished at the lack of quality that many UX Portfolios exhibit. Rarely do I encounter a portfolio that balances esthetics, experience, and technology. The key word in UX is experience and if a candidate’s portfolio has a poor one then we don’t even consider them.

I agree with the article that conveying the rationale behind a final design is important for any project. I believe this UX Portfolio does a great job of doing this, http://www.josearamirez.com. Check it out and see if you agree with me.

One thing that might be useful….if you’re not Google and you don’t have a queue of A* talent beating down the door, it can be useful (budget permitting) to work with a specialist User Experience recruiter. And by specialist I don’t mean generalist IT recruiters, not digital recruiters dabbling in UX, I mean someone who ONLY does UX recruitment and nothing else…. They exist in both the US and the UK.

I disagree with this article, particularly the last sentence - it's making the assumption that those who are just getting wet behind the ears in the UX field can't "bring new, unique skills to your team."

Instead of holding out for UX "superstars" (or rather, UX "unicorns") why not take a competent candidate and mold them into the superstars you desire?

I think the majority of this article is pretty general - i.e. employers have to review resumes, portfolios, and conduct interviews for most new hires.

It would be much more useful if you could give us some insight on what makes hiring a UX professional unique. In my opinion empathy is one of the most critical traits a UX professional needs to have; there are definitely some exercises that can be conducted during an interview to determine if a prospect has a good sense of empathy or not.

I'm not sure that focusing on things like how well someones portfolio is put together makes sense for UX hires. I've consulted with a bunch of companies re: hiring for UX & have found that evaluation of an online portfolio is extremely hard for 3 reasons:
1. Most work that demonstrates UX thinking is confidential
2. There are a lot of fakers who'll use work they had little involvement in
3. It's difficult to know what constraints were in place (i.e. I was forced to do wires in onmigraffle)

If you're going to spend time trying to evaluate an online portfolio, I'm wondering if it wouldn't be a better use of time to just sit down and ask the candidate what parts of the projects he's particularly proud of.

I stopped reading a quarter through because so much of this is painting an inaccurate and grossly undermining example of the variants and types of user experience professionals there are.

You noted that if somebody has a background which is more in research and strategy they wont necessarily have an online portfolio, only to later remark on questioning someones design capabilities and even code!!!

Because I don't know how your article ends, I don't know if anywhere you have said, take the person out to lunch and see whether you actually like them or not. And yes, I absolutely agree that you should be able to provide good results regardless of the project, but again, just because the end site looks not to your liking, how do you know how the candidate made their client feel? How they are at presenting findings or whether people like them or not, do they come off well?

Resumes and portfolios can tell you a lot or absolutely nothing about a person. A 10 minute phone call will give you a far better reflection of somebodies skills than reading something when writing and design may not be their best skills.