The XY problem

I was in Seattle with my boyfriend for Thanksgiving couple years ago. On Wednesday night, I realized mid-dinner I had forgotten to bring my toothbrush and toothpaste. I asked the waitress:

“Do you know if there’s any open convenience store around here?”

The waitress answered: “I don’t know. Sorry.”

“Why did you ask about convenience store?” My boyfriend asked me.

“Because I can buy a toothbrush there.”

“Right, so why didn’t you just ask about the toothbrush?”

“Umm…”

“Have you heard of the XY problem?”

“No…”

“You are trying to solve problem X, and you think solution Y would work, but instead of asking about X when you run into trouble, you ask about Y.”

I get into this XY problem quite frequently in my work. I will often settle on the first solution (the Y) that I think will solve the problem (X) and run around asking questions on StackOverflow asking about Y.

How to avoid XY?

XY problem cannot be solved. It can only be avoided as best as possible and identified as early as possible.

Having said that, there are actually two scenarios when the XY problem occurs. It happens when you are the one facing problem X and when you are the one being asked about solution Y.

Here are the methods I adopt for each scenario

When you are the one with the problem X, slow down

If I had actually written down my description of the situation I was facing, I estimate I would have avoided at least 50% of the XY problems. The key part isn’t necessary about dumping your thoughts on paper. The key part is that writing forces you to slow down.

I know when I slow down, I would be more measured and more likely to come up with multiple solution Y to solve for X. Chances are, the same applies to anyone else. If you don’t like to write it down, simply explaining it verbally with another person helps. This is why some people advocate you talk to rubber ducks when you have an unsolvable problem.

When you articulate your ideas be it verbally or in written form, it forces you to examine more closely what you were previously doing inside your own head

The key point is when you are in your own head, it’s hard to switch between the articulate and examine modes of thinking. Getting your thoughts out of your head gives you breathing space between you and your problem. Allowing new perspectives and fresh thoughts to surface.

Writing allows switching between articulate and examine mode better

So which way is better? Writing or talking? Talking clearly is faster, but sometimes, the number of factors we need to consider are numerous. This is when I suggest writing to be the more powerful option.

Another key advantage that writing has over talking is that you can easily switch between articulate mode and examine mode far better with lower chances of losing your train of thoughts.

We all have this experience. We got stuck in a problem. And then we decided to power through, but the problem refused to yield. After we got stuck for too long, we took a break and walked away. When we came back recharged, suddenly the solution just popped out of nowhere like magic.

Writing down the problem encourages that kind of inspired thinking. Simply because when you describe the problem, you are in one frame of mind. When you come back and read the problem, you may be in a different frame. Different enough that you may solve it way better than when you get stuck on.

When you are the one being asked for a solution, ask why

In my work, often the customers like to ask me, “Can you add this feature?” Then I will inquire deeper why they want to have that feature. This is important because it’s my responsibility to make sure only the right features get added to the systems I build. Most systems crumble under the weight of their own complexity over time because nobody asked, “why do we want to add this?”

Most systems crumble under the weight of their own complexity over time because nobody asked, “why do we want to add this?”

What I noticed is that almost always when this happened, the customer was having a XY-problem moment.

Asking the other person “Why are you asking for this?” or “What are you trying to solve with this request?” helps the other person to slow down and, you guessed it, articulate and examine their own thoughts.

As a consultant, this helps you as well. I can attest to the number of times I turn down bad feature requests without actually rejecting the customer outright. Asking the why questions help them to arrive at the same conclusion. They made up their own mind by convincing themselves. Far superior than you telling them no.

Nobody likes to hear no.

Actual case study of XY problem

I will end off with a recent story that captures this situation which happened recently.

My cousin’s young daughter just started work and has some issues with excel records. Knowing that I help build systems for customers who handle giant excel spreadsheets, she came to me opening with a question about VLOOKUP.

Initial question she asked: how to use VLOOKUP? I asked for the real problem

Now that I realise she wants to create a search box for Excel I asked for more clarifications.

1st level of deeper question she wanted: how to create search box in Excel? I asked more questions

After more questions, it appears that she wanted to perform search within a single column and fetch corresponding record rows.

Which leads me to realise that the existing filter function would work just fine for her use case.

Actual question she really wants to solve: how do I filter records based on 1 column’s data?

Notice her last comment about how she wasn’t aware that filter could solve her problem.

I didn’t know filter can do it

And that’s the other danger of XY problem. You can settle too soon on a preconceived notion of what the solution is for your problem. We all have limited knowledge. Knowing how to articulate and then examine our own formulation of the problem gives us a far better chance of solving it. Of course, it’s also easier to share your formulation of the problem with other people to get advice.

Conclusion: Articulate and Examine your question to avoid XY

When in doubt, simply describe your problem in writing. Employ the articulate and examine frames. Think of yourself when posing the problem in writing and then yourself again when reading it as two different people.

I find this way of treating yourself as two separate people to be so useful it’s almost like an unfair magic trick. To make it easier to have two separate frames is to take a nice 15 min break in between. Go for walk. Have a snack. Go do something manual like cleaning dishes for a short while. Anything that gets you to stop thinking of the problem mentally.

That way, the you before the break would be in the articulate mode. The you after the break would be in the examine mode. If this still fails, at least you now have an almost-ready question to pose back on StackOverflow.