Silicon Valley is the US’ geographic golden child. A glittering, high-tech icon of accomplishment against all odds; a magnet for the innovative and the restless.

Silicon Valley tolerates failure, celebrates it even

Yet this success story comes with a twist: Silicon Valley tolerates failure, celebrates it even. The former orchard groves south of San Francisco are as much an ode to catastrophe and crushing disappointment as they are to achievement.

A culture – some might say a fetish – of failure permeates every open-floor office plan and shiny new Tesla in the Valley. You aren’t considered a bona fide success in Silicon Valley unless you have failed – ideally, multiple times and in spectacular fashion.

There’s even an annual Failcon conference, where people celebrate their failures – and, presumably, learn from them. (Failure, apparently, is contagious; there are now Failcon conferences in Tel Aviv, Bangalore, Barcelona and elsewhere.)

Yes, failure is the engine that drives Silicon Valley; it’s an integral part of the region’s creative ecology – but not for the reasons that are so often touted.

Silicon Valley’s failure fetish traces back to its beginnings, and is intertwined with its geography. It’s no coincidence that the world’s premier technology hub sprouted not in establishment East Coast cities like New York and Boston, but in the far West. California was – and to an extent still is – a place one flees to, a refuge for jilted lovers, bankrupt businessmen, lost souls. As one of the region’s high-tech pioneers, William Forrester of Stratus Computer, put it, “If you fail in Silicon Valley, your family won’t know and your neighbours won’t care.”

And the Valley is, at least in part, a child of the 1960s counter-culture movement, an era when failure, or at least contrariness, was a way of life.

Today, the entire venture capital industry is built around failure. The financiers work on the assumption that the vast majority of their investments, at least 70%, will fail. They’re looking for the unicorns, the blockbuster success that will compensate for all those losses.

Vanity Fair recently compiled a slideshow of 14 of the region’s most spectacular failures, including Theranos, the scandal-plagued biotech company currently under federal investigation. House-cleaning business Homejoy, and RDIO, a music streaming service, came and went so quickly they barely registered. The Valley buries its dead quickly and quietly.

It’s no wonder I have such a hard time finding evidence of the Valley’s past failures – or, for that matter, its successes. Silicon Valley doesn’t do history. History is, at best, an afterthought in a region that has its gaze firmly fixed on the future. Its roots are there, though, provided you are willing to dig a bit.

Determined history pilgrims typically head straight for 367 Addison Avenue in Palo Alto. It’s not the house that interests them but what’s behind it: a small garage with a green door. Here, in 1938, two young Stanford graduates, Dave Packard and Bill Hewlett, spent hours experimenting. They were tinkering. They tried everything: a motor controller for a telescope, a bowling alley device that chirped when someone crossed the foul line, and more – all of them were failures. About a year later, the duo finally happened on a winning invention, an audio oscillator used to test sound equipment, but not until they churned out a series of epic failures.

I sat down one afternoon at a coffee shop in Mountain View, home to Google and boundless optimism. As I sipped my handcrafted, artisanal Ethiopian dark roast, driverless cars glided by silently, sharing road space with the Teslas. My companion was Chuck Darrah, an anthropologist who has spent most of his career studying the strange customs of the Valley inhabitants. Chuck is more observer than participant: skeptical of the Valley’s evangelical belief in the power of technology to improve the world, he doesn’t even own a mobile phone.

One of the biggest myths about Silicon Valley, Chuck told me, is that people here take risks. It is a myth that is simultaneously true and untrue. Silicon Valley celebrates risk, yet at the same time “it has some of the best mechanisms for avoiding the consequences of risk in the world.”

“Such as?” I asked.

“Just think about it. These entrepreneurs, we’re told, deserve their money because of the risk they take. But you don’t see people jumping off the tops of buildings here. They tend to land on their feet. They tend to land in places like this, drinking cappuccinos, because the risk is a peculiar kind of risk. Most of the people in high tech will admit if they lost their job, they would find another one. They might even find a better one.”

“Yes. A huge net. It’s easier to take risk when you are insulated from it.”

Chuck was equally quick to dismiss the old bromide about how the key to success is to embrace failure. The real questions is: what is the difference between failure that leads to innovation, and failure that leads to… more failure?

The answer, researchers now believe, lies not in the failure itself, but how we recall it – or, more precisely, how we store it. Successful failures are those people who remember exactly where and how they failed, so when they encounter the same problem again, even if in a different guise, they are able to retrieve these “failure indices” quickly and efficiently. They are willing to backtrack.

As we finished our coffee, the California sunlight softening to a golden glow, Chuck Darrah explained that the real symbol of Silicon Valley is not the open-plan office or the ping-pong table, but, rather, the moving van. I had spotted one earlier in the day, parked outside a nondescript office block in Mountain View. The movers were busily carting off ergonomic chairs and Danish desks, no doubt discarding the carcass of some failed venture and making room for the next. In Silicon Valley, there is always a next.

If you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Earth, Culture, Capital, Travel and Autos, delivered to your inbox every Friday.