Coincidences and Correlations

by Venkat on July 1, 2013

Most people have heard the admonition, “correlation is not causation.” Few have heard the related admonition, “coincidence is not correlation.”

In common usage, a coincidence is about pairs of rare events that have a background relationship within a model. Like thinking about a friend you haven’t thought about for ten years, and then running into him the next day. But here, I mean a more banal technical sense of “coincidence” — juxtaposition. Co-incidence, as in “occurring together”

So if you and I are at a coffee shop at the same time, our mutual presence is a “co-incidence” whether we are long-lost friends or strangers who ignore each other.

But why are these two admonitions necessary at all? Why would we assume relatedness among unrelated things, or see causal relationships where there are only correlations? They are necessary because decisions enacted in the real world as opposed to inside your head, share physical time and space with other enactments in progress: situations.

Situations can be understood as little bounded regions in space and times through which many narratives flow. In the coffee shop where I am writing this right now, there are several other people doing random other things. Some I’ll never see again and have no relationship to. Others I might see repeatedly and suspect a correlation. Perhaps they live in the same neighborhood. Or their presence on many days in a row might be a string of coincidences with no reason. Maybe they just happened to have several completely different errands in the neighborhood this week.

Correlation might have a shared proximal cause (maybe we live in the same area) or a shared distant cause (I live near here, she works near here). Or there might be actual causation. He’s here because I am here. Maybe somebody trying to steal the secret papers I have in my backpack.

But stepping back, this is a very good way to develop situation awareness of a particular place and time. What are the major rivers of causal flow passing through this time and place? Which ones are just juxtaposed? Which ones run together elsewhere but are unrelated? Which ones interact before and after? Which ones are actually part of the same flow? Which of the many rivers of causation flowing through a time and place actually dominates that situation?

If you’re bad at generally being “in the moment” as I am, this can be a useful way to force yourself out of your own head and parse your surroundings. The huge value to doing this is that you might see connections between event streams that you can make.

Internally focused thinkers often miss opportunities because they are so involved in a particular story, they don’t pay attention to other stories that are evolving alongside. Externally focused thinkers don’t make this mistake. They are able to take advantage of coincidences and correlations.

So while the admonitions may be useful for statisticians analyzing data, they are even more useful for action-oriented people looking for opportunities. You should remember: coincidences can be turned into correlations. Correlations can be turned into causations.

Well, isn’t that basically how scenes based on serendipity evolve? Like startup people working at cafes that develop reputations as “startup hubs” where you overhear gossip, meet potential partners etc.?

“Opportunity is ill-defined” might be one part of a three-part definition of “cultural scene” itself: there is latent, fertile opportunity in a physical time+space that is ill-defined and manifests as coincidences and correlations that can be escalated into relationships and collaborations.

And when extroverts and introverts are in “solution”, it only takes a small shock to “precipitate” each back into their own subgroups. An unstable equilibrium. Once that happens, it’s over for the time being.

I personally believe that in general the equilibrium is so unstable that any pattern of things mysteriously holding together is a sign of a greater than usual concentration of well-developed EN types, or at least those that can play one.

You need E to maintain the bubble and slap fight with other E’s, and you also need N so that I’s feel that their input on things ranging from minutiae to devastating existential death blows are not dismissed due to some kind of E procedural error.

A well-developed E is a plus because they are more aware of the inbuilt limits of E, and in particular, the ES types amongst them. It’s almost like they’re still married but perpetually cheating.

So if you have an abundance of secret EN, you have I connection points to E without I’s having to deal with the tiresome (from their perspective) E protocol of the day.

Another big problem is I’s frequently can’t communicate or end up talking past other I’s. So a good EN can bridge that, provided the I can convince them.

So I think the presence of acting EN types (it doesn’t have to be an actual EN, it can be an IN faking E) is a key binding agent. So I’m not convinced of the serendipity aspect. It’s just too rare. Might as well just fine a couple ENs and simply build it around them.

I find this mostly plausible, except that it feels like an over-extension of Myers-Briggs thinking. In collectives larger than 2, weird new dynamics tend to emerge that are not really “fog” but a different sort of collective beast. I think people’s personalities start to manifest in ways that cannot be reduced to their MBTI styles.

I also think E_F_ is a more important glue template than EN__. Emotions scale to larger groups than thinking.

When you want to go into uncharted waters you have to overextend something or you can’t communicate. This thing on my computer says it’s a file and it’s not even made of paper!

I’m not a huge fan of MBTI or its community, but at least it’s some kind of vocabularity that most people have a passing familiarity with. Kind of like javascript or something.

The pretext here was effectiveness. To take advantage of co-incidental opportunities, you need to be effective in multiple ways. Groups that exist for reasons other than effectiveness are another beast entirely.

For small, non-redundant groups where multidisciplinary thinking is required and effectiveness is a primary driver, somebody playing the effective EN is much more important than IMO the much smaller detail of F/T.