What are insights?

They give us an understanding of a specific cause and effect within a specific context. And help us explain our observations.

We find them subconsciously, and we aren’t actually aware of the chain of inferences that yield insights. Most of the work happens in the background as our brain is making sense of vast quantities of information. [1]

The “Aha!” Moment

Having an insight moves us from a state of not knowing how to solve a problem to a state of knowing how to solve it.[3]

They feel like a lightning bolt and literally give you a hot flush, when you realise “That’s what’s going on here! I had no idea!I was thinking about it the wrong way! And that means that …[!!!]” It’s because bits of existing knowledge align in your head just the right was, and some new observation lights the gunpowder and boom! – all is revealed.[2]

The Problem With Insights

Actively looking for patterns in the data or information you gather takes a lot of time. It’s their subconscious nature that makes them unpredictable, and therefore unreliable in the business context.

When you’re doing research in a commercial setting – the organization’s or client’s objectives are what drives the efforts. You’re never there to just have nice chats with people, you’re there to crack a problem.

In a business context, you usually don’t have weeks or even days of analysing data post-research. So it is difficult to keep digging for the ultimate “why?”

But it’s important to keep our eyes and ears open for new observations, and be open to exploring the issue from multiple angles. [2]

When do we need insights?

We need insights when we are stuck. When we are trying to solve a problem, but are unable to find a solution. [4]

There are three reasons why we can get stuck with these problems[5]:

We misrepresent the problem elements in such a way as to preclude solution;

We focus on information retrieved from memory that is not pertinent to obtaining the solution and may in fact lead us away from the solution; or

We are working with insufficient information to achieve success.

Example: Insight problems in the history of aviation.

#1 Misinterpreting problem elements:

In 1507, John Damian strapped on wings covered with chicken feathers and jumped from the walls of Stirling Castle in Scotland, breaking his thigh; he later blamed it on not using eagle feathers. [10]

In the early 16th century William Dunbar wrote a poem “A Ballad of The False Friar of Tongland, How He Fell in the Mire Flying to Turkey” about it, mocking his attempts. While pillorying this event, Dunbar makes a broader attack on Damian’s character, depicting him as a habitual charlatan. [11]

Perhaps he wasn’t a charlatan, and just acting on a misinterpretation of the problem elements? 🤔

#2 Focusing in the wrong direction:

Until the 19th century many people had postulated that the key to human flight is going to be in understanding how birds fly.

Differences in specific gravity, special construction of the skeleton, and the like, were believed to account for the ability of birds to sustain and alter their position in the air. It wasn’t until Otto Lilienthal, that anyone thought of investigating the structure and mechanics of their wings. [12]

He reframed the problem as such:

At the very first glance we notice that the slender, slightly curved wings execute a peculiar motion, in so far as only the wing-tips move appreciably up and down, whilst the broader arm-portions near the body take little part in this movement, a condition of things which is illustrated in Fig. 76.

Does not this peculiarity show us another means of facilitating flight, and of reducing the necessary expenditure of energy? May we not assume that the comparatively motionless parts of the wings enable the gull to sail along, whilst the tips, consisting of easily rotating feathers, serve to compensate for the loss of forward velocity ? It is unmistakable that the wide Portion of the wing close to the body, which does little work and has little movement, is intended for sustaining, whilst the narrower tips, with their much greater amplitude of movement, have to furnish the tractive power necessary to compensate for the resistance of the bird’s body and for any possible restraining component.

Source: Otto Lilienthal bird flight as a basis of aviation, New York 1911

His insight led to the glider experiments which were the foundation of the work for the Wright Brothers. As they figured out how to make human flight possible.

# 3 Not knowing enough

Finally, when the Wright brothers started their experiments they wrote the following in a letter to the Smithsonian:

The Smithsonian Institution, Washington:

Dear Sirs:

“I am an enthusiast, but not a crank in the sense that I have some pet theories as to the proper construction of a flying machine. I wish to avail myself of all that is already known and then if possible add my mite to help on the future worker who will attain final success.”

–Wilbur Wright

The believed that careful study of aerodynamics and control with simple hang gliders, would lead to a more sophisticated powered airplane capable of sustained, controlled flight.

Their insight: if the wing on one side of the aircraft met the oncoming flow of air at a greater angle than the opposite wing, it would generate more lift on that side. In response, that wing would rise, causing the aircraft to bank. If the pilot could manipulate the wings in this way, he could maintain balance and turn the aircraft as well.

An insight problem requires us to shift our perspective and view the problem in a novel way in order to achieve the solution. They are different from non-insight problems, where the solution can be constructed incrementally based on our existing mental models. [4]

To overcome these, we have to break our line of reasoning allowing us to shift our solution strategy and find new paths to the solution. [6]

What does it take to find Insights?

People who are most successful at solving insight problems develop a representation of the problem and apply heuristics to transform the problem space so that it looks like the solution space. Eventually, when progress stops, they apply the “restructure when stuck” heuristic. [6]

Solving insight problems requires a deeper conceptual understanding allowing us to select relevant knowledge components and combine them in novel ways, or to guide us to attend to the environmental stimuli that are most relevant. [8]

That’s why it gets easier to find new insights as you develop expertise in a particular area. [9]

Three ways to get better at finding insights.

The three ways you can get better at finding insights are [1]:

Gather new information:Exposing ourselves to a shitload of information from many different sources. You can’t just do this within the context of work, because the volume and quality of the information you get from work pales in comparison to the information you get outside of it. Live curiously and openly and diversely. Have hobbies. Surround yourself with different kinds of people. Travel. Read. Watch good movies.

Actively process new information:Learn how to process information very well. Not through affinity diagramming and cute design techniques, as helpful as they can be, but through epistemology, logic, metaphysics and reflection. Fundamentals are more important than techniques, and we often neglect them.

Communicate rationale:Get better at communication by talking to lots of kinds of people. When you can’t rely on jargon and context specific conventional wisdom, you’re forced to articulate things properly. This articulation helps you find relationships in your observations so you can explain them better.

The three practices above can help us develop a deep and complete understanding of what’s going on – and keep us attuned to what we’re looking to understand and why.

The subconscious flashes of inspiration seem to start coming more reliably then. Developing these mental models doesn’t rely on insights, but they can help to generate ideas to test and explore more easily – and lead to those insights.

How to find insights in practice.

The key takeaway is to note that finding insights depends on the mental models we frame – i.e. how we store and organize information in our brain. One of the problems is that these models are hidden from us, and it isn’t easy to keep track of everything we learn/observe.

We’ve been working to solve this problem by thinking of simple ways to categorize and organize observations in the data we gather – so we can easily access and analyse our mental models. As well as share them with everyone else so we can refine those models and get better at finding insights faster.

Last week I shared my analysis of what it takes to find insights, and I received very helpful comments from experts in the field. They highlighted areas in my previous article which I hadn’t covered.

I took their feedback and synthesized it to develop a more refined understanding of the subject. This article serves to share those insights.

Here are the steps:

I copied all the thoughtful comments which I received.

I took note of the key points in each one of them.

I categorized those points.

I identified the key themes in them.

These themes are the titles of each of the sections above.

Once I had those themes, I identified the areas in which I couldn’t substantiate.

I looked for references from journal papers in psychology about insights.

I put them together, and I’m sharing them with you.

I did all this in three hours this morning by dogfooding our product. We’re building it to simplify how to organize and make sense of qualitative data, find and share insights.

We’re currently doing a closed beta, and would love to have you guys try it out and get your feedback on whether we’ve able to simplify finding insights in a meaningful way.

It has taken us several rounds of testing to get into a shape where it is simple to use. The solution is based on the principles from Grounded Theory, and Thematic Analysis and designed to help make sense of observations from qualitative data.

Further reading.

You can read more about how doing thematic analysis helps in making sense of qualitative data over here.