Friday, September 27, 2019

An exchange of messages with Sabine Hossenfelder about the Many Worlds Interpretation (MWI) of quantum mechanics has helped me sharpen my view of the arguments around it. (Sabine and I are both sceptics of the MWI.)

The case for Many Worlds is well rehearsed: it relates to the “measurement problem” and the idea that if you take the “traditional Copenhagen” view of quantum mechanics then you need to add to the Schrödinger equation some kind of “collapse postulate” whereby the wavefunction switches discontinuously from allowing multiple possible outcomes (a superposition) to having just one: that which we observe. In the Many Worlds view postulated by Hugh Everett, there is no need for this “add on” of wavefunction collapse, because all outcomes are realized, in worlds that get disentangled from one another as the measurement proceeds via decoherence. All we need is the Schrödinger equation. The attraction of this idea is thus that it demands no unproven additions to quantum theory as conventionally stated, and it preserves unitarity because of the smooth evolution of the wavefunction at all times. This case is argued again in Sean Carroll’s new book Something Deeply Hidden.

One key problem for the MWI, however, is that we observe quantum phenomena to be probabilistic. In the MW view, all outcomes occur with probability 1 – they all occur in one world or another – and we know even before the measurement that this will be so. So where do those probabilities come from?

The standard view now among Everettians is that the probabilities are an illusion caused by the fact that “we” are only ever present on one branch of the quantum multiverse. There are various arguments [here and here, for example] that purport to show that any rational observer would, under these circumstances, need to assign probabilities to outcomes in just the manner quantum mechanics prescribes (that is, according to the Born rule) – even though a committed Everettian knows that these are not real probabilities.

The most obvious problem with this argument is that it destroys the elegance and economy that Everett’s postulate allegedly possesses in the first place. It demands an additional line of reasoning, using postulates about observers and choices, that is not itself derivable (even in principle!) from the Schrödinger equation itself. Plainly speaking, it is an add-on. Moreover, it is one that doesn’t convince everyone: there is no proof that it is correct. It is not even clear that it’s something amenable to proof, imputing as it does various decisions to various “rational observers”.

What’s more, arguments like this force Everettians to confront what many of them seem strongly disinclined to confront, namely the problem of constructing a rational discourse about multiple selves. There is a philosophical literature around this issue that is never really acknowledged in Everettian arguments. The fact is that it becomes more or less impossible to speak coherently about an individual/observer/self in the Many Worlds, as I discuss in my book Beyond Weird. Sure, one can take a naïve view based on a sort of science-fictional “imagine if the Star Trek transporter malfunctioned” scenario, or witter on (as Everett did) about dividing amoebae. But these scenarios do not stand up to scrutiny and are simply not science. The failure to address issues like this in observer-based rationales for apparent quantum probabilities shows that while many Everettians are happy to think hard about the issues at the quantum level, they are terribly cavalier about the issues at the macroscopic and experiential level (“oh, but that’s not physics, it’s psychology” is the common, slightly silly response).

So we’re no better off with the MWI than with “wavefunction collapse” in the Copenhagen view? Actually, even to say this would be disingenuous. While some Everettians are still happy to speak about “wavefunction collapse” (because it sounds like a complicated and mysterious thing), many others working on quantum fundamentals don’t any longer use that term at all. That’s because there is now a convincing and indeed tested (or testable) story about most of what is involved in a measurement, which incorporates our understanding of decoherence (sometimes wrongly portrayed as the process that makes MWI itself uniquely tenable). For example, see here. It’s certainly not the case that all the gaps are filled, but really the only thing that remains substantially unexplained about what used to be called “collapse” is that the outcome of a measurement is unique – that is, a postulate of macroscopic uniqueness. Some (such as Roland Omnès) would be content to see this added to the quantum formalism as a further postulate. It doesn’t, after all, seem a very big deal.

I don’t quite accept that we should too casually assume it. But one can certainly argue that, if anything at all can be said to be empirically established in science, the uniqueness of outcomes of a measurement qualifies. It has never, ever been shown to be wrong! And here is the ultimate irony about Many Worlds: this one thing we might imagine we can say for sure, from all our experience, about our physical world is that it is unique (and that is not, incidentally, thrown into doubt by any of the cosmological/inflationary multiverse ideas). We are not therefore obliged to accept it, but it doesn’t seem unreasonable to do so.

And yet this is exactly what the MWI denies! It says no, uniqueness is an illusion, and you are required to accept that this is so on the basis of an argument that is itself not accessible to testing! And yet we are also asked to believe that the MWI is “the most falsifiable theory ever invented.” What a deeply peculiar aberration it is. (And yet – this is of course no coincidence – what a great sales hook it has!)

Sabine’s objection is slightly different, although we basically agree. She says:

“Many Worlds in and by itself doesn't say anything about whether the parallel worlds "exist" because no theory ever does that. We infer that something exists - in the scientific sense - from observation. It's a trivial consequence of this that the other worlds do not exist in the scientific sense. You can postulate them into existence, but that's an *additional* assumption. As I have pointed out before, saying that they don't exist is likewise an additional assumption that scientists shouldn't make. The bottom line is, you can believe in these worlds the same way that you can believe in God.”

I have some sympathy with this, but I think I can imagine the Everettian response, which is to say that in science we infer all kinds of things that we can’t observe directly, because of their indirect effects that we can observe. The idea then is that the Many Worlds are inescapably implicit in the Schrödinger equation, and so we are compelled to accept them if we observe that the Schrödinger equation works. The only way we’d not be obliged to accept them is if we had some theory that erases them from the equation. There are various arguments to be had about that line of reasoning, but I think perhaps the most compelling is that there are no other worlds explicitly in any wavefunction ever written. They are simply an interpretation laid on top. Another, equally tenable, interpretation is that the wavefunction enumerates possible outcomes of measurement, and is silent about ontology. In this regard, I totally agree with Sabine: nothing compels us to believe in Many Worlds, and it is not clear how anything could ever compel us.

In fact, Chad Orzel suggests that the right way to look at the MWI might be as a mathematical formalism that makes no claims about reality consisting of multiple worlds – a kind of quantum book-keeping exercise, a bit like the path integrals of QED. I’m not quite sure what then is gained by looking at it this way relative to the standard quantum formalism – or indeed how it then differs at all – but I could probably accept that view. Certainly, there are situations where one interpretational model can be more useful than others. However, we have to recognize that many advocates of Many Worlds will have none of that sort of thing; they insist on multiple separate universes, multiple copies of “you” and all the rest of it – because their arguments positively require all that.

Here, then, is the key point: you are not obliged to accept the “other worlds” of the MWI, but I believe you are obliged to reject its claims to economy of postulates. Anything can look simple and elegant if you sweep all the complications under the rug.

It would seem perverse, almost rude, not to begin a discussion of imagination in physics with Einstein’s famous quote on the topic, voiced during a newspaper interview with the writer George Viereck in 1929:
“I'm enough of an artist to draw freely on my imagination, which I think is more important than knowledge. Knowledge is limited. Imagination encircles the world.”

For a fridge-magnet inspirational quote to celebrate the value of imagination, you need look no further. But context, as so often with Einstein, is everything. He said this after talking about the 1919 expedition led by the British physicist Arthur Eddington to observe the sky during a total solar eclipse off the coast of Africa. Those observations verified the prediction of Einstein’s theory of general relativity that starlight would be bent by the gravitational field of a massive body like the sun. Einstein told Viereck that “I would have been surprised if I had been wrong.” Viereck – a fascinating figure in his own right, who had previously interviewed (and showed some sympathy for) Adolf Hitler and wrote a psychological and gay-inflected Wildean vampire novel in 1907 – responded to that supremely confident statement by asking: “Then you trust more to your imagination than to your knowledge?”

You could say that Einstein’s reply was a qualified affirmative. And this seems very peculiar, doesn’t it, for a “man of science”?

The story dovetails with Einstein’s other well-known response to the eclipse experiment. Asked by an assistant (some say a journalist) how he should have felt if the observations had failed to confirm his theory, he is said to have responded “Then I would feel sorry for the dear Lord. The theory is correct.”

Compare that with the statement of another celebrated aphoristic physicist, the American Richard Feynman:
“It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong.”

Who is right? Einstein trusting to imagination, intuition and artistry, or Feynman to the brutal judgement of empiricism? If we’re talking about scientific methodology, Feynman is right in spirit but nonetheless displaying the limitations of the physicist’s common “naïve realist” position about science, which assumes that nature delivers uncomplicated, transparent answers when we put to it questions about our physical theories. Yet Einstein’s general relativity was a theory so profoundly motivated and so conceptually satisfying, despite the mind-boggling shift it demanded in conceptions of space and time, that it could not be lightly tossed on the scrapheap of beautiful ideas destroyed by ugly facts.

So the sensible way to have handled a discrepancy with observed “facts” like those collected by Eddington in his observations of the positions of stars during an eclipse would have been to wonder if the observations were reliable. Indeed, Eddington was later accused of cherry-picking those facts to confirm the theory, perhaps motivated by his Quaker’s desire to bring about international reconciliation after First World War had triggered the ostracising of Germany. (It seems those charges were unfounded.) Nature doesn’t lie, but experimentalists can blunder.

*

There’s a deeper reason to valorize Einstein’s claim about imagination in physics. What I feel he is really saying is that imagination precedes knowledge, and indeed establishes the precondition for it. You might say that when the shape of imagination sufficiently fits the world, knowledge results.

We have never needed more reminding of this. In his unfinished magnum opus Novum Organum the seventeenth-century English philosopher Francis Bacon presented knowledge as the product obtained when raw facts – observations about the world – are fed into a kind of science machine (what we might now call an algorithm) and ground into their essence. It was an almost mechanical process: you first collect all the facts you can, and then refine and distil them into general laws and principles about the way the world works. Bacon never completed his account of how this knowledge-extraction process was meant to work, but at any rate no one in science has ever successfully used such a thing, or even knows what it could comprise.

Yet Bacon’s vision threatens to return in an age of Big Data – especially in the life sciences, where the availability of information about, say, genome sequences or correlations between genes and traits has outstripped our ability to create theoretical frameworks to make sense of it. There’s a feeling afoot not only that data is intrinsically good but that knowledge has no option but to fall out of it, once the mass of information about the world is large enough.

Physicists have received advance warning of the limitations of that belief. They have their own knowledge machines: sophisticated telescopes and particle detectors, say, and most prominently the Large Hadron Collider and other particle colliders capable of generating eye-watering quantities of data about the interactions between the fundamental constituents of the world. But they already know how little all this data will help without new ideas: without imagination.

For example? There are some good reasons to believe that if physics is going to penetrate still further into the deep laws of nature, it needs a theoretical idea called supersymmetry. So far, all we know about the particles and forces of nature is described by a framework called the Standard Model, which contains all the ingredients seemingly needed to explain everything seen in experiments in particle physics to date. But we know that there’s more to the universe than this, for many reasons. For one thing, the current theory of gravity – Einstein’s general relativity – is incompatible with the theory of quantum mechanics used to describe atoms and their fundamental particles. Supersymmetry – a putative connection between two currently distinct classes of particle – looks like a promising next step to a deeper physics. Yet so far, the LHC’s high-energy collisions have offered no sign that it’s true.

What’s more, there’s nothing in the Standard Model that seems to account for the “dark matter” that astrophysicists need to invoke to explain what they see in the cosmos. This mysterious substance is believed to pervade the universe, invisibly, being felt by ordinary matter only through its gravitational influence. Without something like dark matter, it is hard to make sense of the observed forms and motions of galaxies: how they rotate without shedding stars like a water sprinkler. The reasons to believe in dark matter – and moreover to believe it exceeds the mass of ordinary visible matter by a factor of about five – are very strong. Yet countless efforts to spot what it consists of have failed to offer any clues. Huge quantities of data constrain the choices, but no evidence supports any of the theories proposed to explain dark matter.

These are – there is no avoiding the issue – failures of imagination. Supersymmetry and dark matter are wholly imagined theories or entities, but the collective imagination of physicists has not yet made them vivid enough to be revealed or disproved. It is possible that this is because they are imaginary in the more literary sense: they exist only in our minds. And they are not alone; dark energy (which causes the universe to expand at an increasing rate) and string theory (one candidate for a theory that would unite gravity and quantum mechanics) are other components of the physicist’s imaginarium waiting to be verified and explained or to be dismissed as unicorns, as the ether, as the philosopher’s stone.

A single observation – one experiment revealing a discrepancy with a definite theoretical prediction, or one sighting of a new kind of particle – could change the situation. Maybe it will. But it is equally possible that we will need ultimately to concede defeat, and to extent the imagination of physics into new territory: for example to accept, as some are already arguing, that what we call “dark matter” is a symptom of another physical principle (a modification to the theory of gravity, say) and not a true substance.

There’s nothing embarrassing or damning in all this. It’s not that physics itself is failing. The situation is just business as usual in science: to have mysteries awaiting explanation, even ones of this magnitude, is a sign of health, nor sickness. For individual physicists whose reputations hang (or seem to) on the validity of a particular idea, that’s scant comfort. But for the rest of us it’s nothing short of exhilarating to see such deep and broad questions remaining open.

*

The real point is that imagination in physics is what the paths to the future, to new knowledge, are built from. Actual knowledge – things we can accept as “true”, in the sense that they offer tried and tested ways of predicting how the world behaves – has been assembled into an edifice as wonderful and as robust as the Gothic cathedrals of stone, the medieval representations of the physical and spiritual universe. But at the point where knowledge runs out, only imagination can take us further. I think this is what Einstein was driving at.

The invitation is often to suppose that this imagination operates only at the borders of physical theory: at, you might say, the cliff-face of physics that tends to dominate its public image, where we find exotica like string theory, black holes, cosmology and the Higgs boson. But physics, perhaps more than any other science, has a subtle, fractal-like texture in which gaps in knowledge appear everywhere, at all scales. Imagination was needed to start to understand that strange state of matter made of grains: powders and sand, part fluid and part solid. It is currently blossoming in a field known as topological materials, in which the electrical and magnetic properties are controlled by the abstract mathematical shapes that describe the way electrons are distributed, with twists akin to those in the famous one-sided Möbius strip. It was imagination that prompted physicists and engineers to make structures capable of acting as ‘invisibility shields’ that manipulate and guide light in hitherto inconceivable ways. In all these cases, as in science more broadly, the role if the imagination is not so much to guide us towards answers as to formulate interesting and fruitful new questions.

What does this imagination consist of? We’d do well to give that question more attention. I would suggest that it is, among other things, a way of seeing possibilities: a rehearsal of potential worlds. That’s what justifies Einstein’s comparison to the work of the artist: imagination, as Shakespeare put it, “bodies forth the forms of things unknown.” The scientist’s theories, as much as the poet’s pen, “turns them to shapes and gives to airy nothing a local habitation and a name.” That name could be “general relativity” – why not?

What’s the source, though? Many ideas in fundamental physics grow from what might seem the rather arid soil of mathematics. Supersymmetry and string theory are predicated in particular on the conviction that the deepest principles of the physical world are governed by symmetry. What this word means at the level of fundamental theory might seem less apparent to the outsider than what it implies in, say, the shape of a Grecian urn or the pattern of wallpaper, but at root it is not so very difference: symmetry is about an equivalence of parts and their ability to be transformed one into another, as a left hand becomes a right through the mirror reflection of the looking glass.

Well, it might seem arid, this mathematics. But imagination is as vital here as it is in art. What mathematicians value most in their colleagues is not an ability to churn out airtight proofs of abstract theorems but a kind of creativity that perceives links between disparate ideas, an almost metaphorical way of making connections in which intuition is the architect and proof can come later. Both mathematicians and theoretical physicists commonly speak of having a sense that they are right about an idea long before they can prove it; that proof is “just the engineering” needed to persuade others that the idea will hold up.

Let’s be cautious, though, about making “engineering” the prosaic, plodding part of science though. The common perception is that theorists do the dreaming and experimentalists just build the apparatus for putting dreams to the test. That’s just wrong. For one thing, it’s typically experiment that drives theory, not the other was around: it’s only when we have new instruments for examining the world that we discover gaps in our understanding, demanding explanation. What’s more, experiment too is fuelled by imagination. No one tries to see something unprecedented – farther out into space (which means, because light’s speed is finite, farther back in time), or into the world of single atoms, or into the spectrum of radiation outside the band of light our eyes can register – unless they have conjured up images of what might be there. Sure, you need some existing theory to guide your experimental goals, to show potentially fruitful directions for your gaze; but no one sails into uncharted territory if they think all they’ll find is more of the same, or nothing at all. “If you can’t imagine something marvellous, you are not going to find it”, says physics Nobel laureate Duncan Haldane. “The barrier to discovering what can be done is actually imagination.” And the power and artistry of the experimenter’s imagination comes not just from dreaming of what there is to be found in terra incognita, but also from devising a means to travel there.

When I speak of dreams, I don’t just mean it metaphorically, nor just in the sense of waking reverie. To judge from the testimony of scientists themselves, dreams can function as sources of inspiration. True, we should be a little wary of that; the notion of receiving insight in a dream became a romantic trope in the nineteenth century, and careful historical analysis often reveals some hard and very deliberate graft, as well as a very gradual process of understanding, behind scientific advances that were recast retrospectively as dream-revelations. But it happens. Several contemporary physicists have attested to insights that came to them in dreams, as the conscious mind that has been long pondering a problem loosens its bonds on the margins of sleep and admits a little more of the illogic on which imagination thrives.

All the same, we shouldn’t think that the physicist’s imagination always works in the abstract, in the realm of pure thought. Very often, it takes visual form: finding the right symbolic representation of a problem, such as Feynman’s famous “diagrams” for studying questions in the field of quantum electrodynamics (in essence, the theory of how light and matter interact), can unlock the mind in ways that more abstract algebraic mathematics or calculus can’t. Pen and paper can be the fuel of the imagination. As Cambridge physicist Michael Cates (incumbent of the chair previously held by Stephen Hawking and Isaac Newton) has said, “I need a piece of paper in front of me and I’m pushing symbols around on the page… so there’s this interaction between processing in your head and moving symbols around.” Never underestimate the traditional blackboard as a tactile, erasable aid to the imagination. The productivity of such aids is no surprise. Ask a child to think of a story, and it’s ne easy matter. Give them a doll’s house full of figurines, and they’re away.

*

Yet whether it is theoretical or experimental, this imagination in science (as in art) is not idle fantasy. It is a condensation of experience: it takes what you know and plays with it. I do mean “plays”: imagination is nothing if not ludic. But it is also the very stuff of thought. One interpretation of cognition, in the context of artificial intelligence, is that it is largely about figuring out the possible consequences of actions we might make in the world: an “inner rehearsal” of imaginary future scenarios. Imagination in science extends that process beyond the self to the world: given that we know this, mightn’t things also be arranged like that?

It’s much more than a guess, then, and as Shakespeare hints, has almost the power of an invocation. Truly, the scientific imagination can invoke into being something that was not there before. Isaac Newton was cautious about his “force of gravity”, knowing that he risked (and indeed incurred from his arch-rival Gottfried Leibniz) accusations of occultism. Yet all the same this “force” became – and remains – a ‘thing’ in physics, even if we can regard it as a figure of speech, a convenient conceptual tool that general relativity invites us to regard otherwise as curvature of spacetime. It’s a process entirely analogous to the way Shakespeare goes on to speak, in A Midsummer Night’s Dream, of how correlation leads us to imagine causation:
Such tricks hath strong imagination,
That if it would but apprehend some joy,
It comprehends some bringer of that joy.

In this way we’re reminded that imagination shares the same etymological root as “magic” – which, in the age just before the time of Isaac Newton, did not necessarily mean superstitious agency but the “hidden forces” by which natural magicians comprehended and claimed to manipulate nature. In that regard Newton wasn’t, as John Maynard Keynes claimed, the “last of the magicians”, in the sense of his having a belief in occult forces (such as gravity, acting invisibly across space). No, if that was Newton’s “magic” then today’s physicists share a conviction in it, for any model in physics awards imagination this role of employing imagined causative agencies – things we might not perceive directly but which manifest through their effects, such as dark matter, dark energy, or the Higgs field – to explain what we see.

Now, though, physics places demands on the imagination as never before. I’m struck by how dark matter and dark energy, say, commandeer known concepts (mass, energy) that may or may not turn out to be appropriate. Even more challenging are efforts to provide some physical picture of quantum mechanics, the kind of physics generally used to describe atoms and fundamental particles. These objects don’t seem to conform to our intuitions derived from the everyday world of rocks and stones, tennis balls and space rockets. They can, for example, sometimes display behaviour we associate not with particles but with waves. They appear to be able to influence one another instantaneously over long distances; they are said to exist “in several states or places at once.”

Yet these descriptions are attempts – often clumsy, sometimes misleading – to make quantum mechanics fit into the forms of our conventional “classical” imagination. Arguments and misperceptions follow, or a disheartening decision to draw a veil over quantum improprieties by calling them “weird”. We can and should do better, but this will require a reshaping, an expansion, of our imaginative faculties. We have to develop a kind of intuition that is not constrained by our daily experience – because if there’s one thing we can be sure about in quantum mechanics, it’s that it demands the possibility of phenomena that lie outside this experience.

To venture into unknown territory, where imagination is at a premium, is a risk. To put it bluntly, your imagination is more likely to lead you astray than toward the truth. It is no magical guarantor of insight. Will you take that risk? Mathematical physicist Jon Keating has put the problem succinctly: “[How can we] encourage people to make them feel more comfortable with the failure that comes with most creative and imaginative ideas?” Unless we get better at that, educationally or institutionally, science will suffer.

And it’s very possible that physicists won’t alone accomplish the feats of imagination needed to crack their hardest problems. They may need to find inspiration from philosophy, art, literature, aesthetics. Imagination doesn’t recognize categories and boundaries – it is a power that aims to encircle the world.