Q: How is it that Bell’s Theorem proves that there are no “hidden variables” in quantum mechanics? How do we know that God really does play dice with the universe?

Physicist: Bell’s theorem, and its philosophical fallout, is one of the most profound discoveries since relativity.

Bell’s theorem states (among other things) that the universe is fundamentally unpredictable, and that quantum mechanical things (for example: everything) are not actually in one state. If a box could contain either a blue marble or a red marble, then when you open it you’ll see either one or the other. In “reality” it was one color or the other before you open the box. In QM, it can be both before you open the box (it’s actually still both afterwards, but moving on…).

Einstein (and most other physicists of the time) believed that if you knew everything about a system of particles (no matter how big) that you could theoretically predict what that system will be doing in the future, perfectly. Homeboy thought that the only reason that the movement of air molecules seems to be random, is that we can’t perfectly measure that exact position and velocity of every single one. So he thought that every particle truly is in some particular state, but that we merely don’t know for sure what that state is (the marble in the box has only one color, but we don’t know what it is).

The idea that randomness and unpredictability are caused by unknown (or unknowable) things is called “hidden variable theory” (The ‘Stein believed in this). For example; 2, 2, 3, 6, 0, 6, 7, 9, 7, 7, 4, 9, 9, … is not random, but seems random. It would be really hard to predict the next term (7) if you don’t know the hidden variable. (BTW, the “hidden variable” is: this is the decimal expansion of )

Bell’s theorem essentially boils down to a proof that the result of an experiment doesn’t exist until the measurement is made (so it can’t be predicted). Hidden variable theory presupposes that the particles involved are in definite states, which means that the result of a measurement already exists before the measurement is made. For example: before you open a gift what you’ll see is already set in stone. The gift is a set thing before you open the box. This is not the case for most quantum mechanical systems.

Here’s one of the experiments that demonstrates Bell’s theorem, and two ways to look at it.

An entangled pair of photons is created and fired in opposite directions. En route the polarizers are randomly oriented, then the detectors measure whether or not the photons pass through. This is done hundreds of thousands of times to measure the relationship between 1) the difference in angles between the polarizers and 2) the probability of measuring the same result.

The experiment: Step 1: Create a pair of entangled photons and fire them in opposite directions. Entangled particles always yield the same result when they are subjected to the same measurement, and are likely to yield the same result for similar measurements.

Step 2: Randomly orient the polarizers, after the entangled pair is created, but before either is detected (this is hard to time, and is really fast). This is done so that the photons “don’t know what to expect” and “can’t compare notes”. Information about polarizer A would have to travel faster than the speed of light to get to photon B before photon B hits it’s own polarizer. So, without faster than light effects (which don’t exist for many, really good reasons) the photons are each acting independently. The orientation is random so that the photons can’t “plan ahead”.

Step 3: Measure the polarization. If the detector “clicks” then the photon made it through the polarizer, and therefore has the same polarization. If the detector doesn’t click, then the photon had the opposite polarization and was stopped.

The probability of the measurements being the same (for an entangled pair) is , where is the difference in angles between the polarizers. It is tricky to see why, but this probability is impossible if you assume that the result of a measurement exists before the measurement is made. Here’s why.

The possible polarizations for polarizer A (red) and polarizer B (blue).

Algebraic approach: Restricting the possible angles of the polarizers to 0° and 45° for A, and 22.5° and 67.5° for B, run the experiment. Here’s what’s about to happen:

1) If you could predict the outcome of each version of the experiment, then you could find a definite value of L (see below).

2) For strictly (unarguable) mathematical reasons L = ±2.

3) Experimentally we find that the average value of L is 2√2.

4) But this is a contradiction, so we cannot actually make useful predictions.

Now it’s happening:

If polarizer A is at 0° and the detector clicks then you’d say “A0 = 1″, and if the detector doesn’t click then “A0 = -1″. Similarly, you can define B67.5, A45, and B22.5. Just for the hell of it, take a look at: L = A0B22.5 + A45B22.5 + A45B67.5 – A0B67.5 = (A0 + A45)B22.5 + (A45 – A0)B67.5

However, you can’t make all of these measurements simultaneously, so you can’t actually get A0B22.5 + A45B22.5 + A45B67.5 – A0B67.5 for each run of the experiment. The best you can do is find one of these four terms each time you run the experiment. For example, if the polarizer A was randomly set to 45° and the detector clicked, and polarizer B was randomly set to 22.5° and the detector didn’t click, then you just found out that A45B22.5 = (1)(-1) = -1 for that run.

You can however find the expectation value by running the experiment over and over and keeping track of the results and polarizer orientation.

So E[A0B22.5] + E[A45B22.5] + E[A45B67.5] – E[A0B67.5] ≤ 2. This is one version of “Bell’s Inequality”, and it holds if each term (A0, A45, B22.5, B67.5) has a value.

Using the fact that the chance of getting the same result is , and that each term is 1 when the results are the same ((1)(1) or (-1)(-1)), and -1 when the results are different ((1)(-1) or (-1)(1)), you can calculate each term. For example:

You’ll find that:

Holy crap! ! But that’s a violation of Bell’s inequality! But the existence of each measurement (whether or not you actually do that measurement) is all you need for Bell’s inequality! So if the inequality is false, then the result of those measurements don’t exist if the measurement isn’t made!

Maybe, if you’re clever and have ready access to a time machine, you could go back and do all the measurements you didn’t make the first time. Then all the results would have to exist! They’d just have to!

Me and my time machine vs. quantum mechanics: If the results exist, but you just didn’t happen to do all the measurements, why not get a time machine? Then you could do one measurement, go back, do a different measurement, go back, do a different measurement, … Then every possible result would be known.

However, once again that correlation probability () screws things up.

So, for example, if the photon goes through at 50°, and then you go back in time, change the polarizer to 51°, and repeat the experiment, then there’s a 99.97% chance (cos2(1°) = 0.9997) that the photon will go through again.

One result from probability says that . Do this twice and you get . So if you measure in the 0° direction to find A0, then go back and change the angle by 1° and repeat this until you’re measuring at 90°, then:

So, if you go back and forth in time to measure whether or not the photon goes through at 1° increments, then there’s a 97% chance that by the time you get to 90° you’ll be getting the same result you did at 0°. However, in reality .

But this is a contradiction. So the results of each measurement (A0, A1, A2, …, A90) can’t all exist.

If I had to guess, every time you go back in time the experiment is completely reset, and the experiment becomes completely random again. The reason (such as it is) is below this unsettling picture.

Wait. Wait… Why?

But why?!: It turns out that the reason that the results of a quantum event can’t be predicted, is that every possible result of that event plays out. So if you ask “will I see the photon go through the polarizer?” the answer is “yes, some versions of you will see the photon go through” and an equally valid answer is “no, some versions of you will not”.

If different versions of you will see every possible result, then the result can’t be predicted, and doesn’t really exist one way or the other until after the measurement is done. At that time the different versions of you will disagree on the result. But don’t worry too much. You’ll never meet you’re parallel-universe twins.

There’s a subtle, but important, difference between “interaction” and “measurement”. For example, sugar water affects the right and left circular polarizations of light slightly differently, with the effect that rotates the linear polarization. So the sugar water interacts with the photons, but no measurement is ever made. The amount of knowledge and the lack of knowledge you have about the photon before and after passing through the sugar water stays the same.
In a polarizer, the light certainly needs to pass through it, and so will interact with the constituent matter (air, plastic, whatever). However, the “polarizerness”: that light with one polarization is blocked, while light with the opposite polarization passes through, is absolutely a measurement. Whether or not light passes through tells you something about the photon’s polarization that you may not have known before.

It always has both, just not a definite value of either. The uncertainty principle applies on all scales, but you’d never notice it on a large scale. The errors implied by the uncertainty principle are really, really, crazy small. But they’re not zero.

“The errors implied by the uncertainty principle are really, really, crazy small. But they’re not zero.”

I just posted in another article of yours referring to basically the same thing. I’m not a physicist, which may be obvious, but as an outsider it is hard not to think that a lot of modern physics is what I would call “mistaking the map for the road.”

The Uncertainty Principle, for instance: just because we do not possess tools that can measure location and velocity perfectly at the same time is not proof that the particle actually sort-of-exists in a probability cloud. A probability cloud may be a great tool for describing the results we get when observing an electron, but that should not be taken to assume that it represents the ultimate reality of what is going on. Just because we cannot prove or know exactly where it is and at what velocity is not proof that it doesn’t have a true location and velocity.

The idea that light is both a particle and a wave (forgive me if this is no longer a relevant concern, but I know that it was at one time)…seriously, has it never occurred to anyone that it is neither and that particles and waves are artifacts of human perception, even (or perhaps especially) when that perception is filtered through millions of dollars of instruments?

This is not some magical case of reality being simultaneously in two contradictory states at one time–this is the result of our failure to fully grasp the ultimate reality, which is quite probably not self-contradictory if we could see it for what it is.

My guess it that the “hidden variables theory” is right on the money, and furthermore that its basic concept is more broadly applicable than was originally intended.

I have the feeling that a whole host of the things that have been mathematically “proven” in the last few decades will someday be seen to have no relation to the reality they are attempting to model.

What Bell’s Theorem shows (in the article above) is that “local hidden variables” are inconsistent with experiment (impossible). This means that either 1) thing conspire with each other in a very mysterious, faster-than-light way (this is non-local), or 2) things are not in a definite state (no hidden variables).
There’s a lot of very direct, straightforward experimental evidence for many-stateness (for example, in fact there’s even a post that talks about one example!), and also a lot of both theoretical and experimental evidence showing that faster than light stuff is impossible.
Physicists are pretty good about tying theory to physical experiment, and at the end of the day physical evidence always wins (even if it messes everything up, or makes you look bad). So, while quantum theory may be a pretty set of theories and equations, if they weren’t all thoroughly backed up by experiment after experiment, they would have been abandoned.
With regard to the uncertainty principle, it is a mathematically derived statement, but it’s based on very sound physical principles, and it’s stood the test of time (and also tests). It’s just one manifestation of “many-statedness”.

It’s taken me a very long time to wrap my head around Bell’s Inequality, but I finally have, thanks to some simplified math I found off in some corner of the internet. (My math is… bad. I can’t math.). My request is: can you write an article that explains it in a way that could be understood with just a good grasp of algebra? It’s such an important cornerstone of physics, and it goes ignored/misunderstood mostly because of that, I think, when really, it’s a very simple thing.

After decades of trying, no experiment has been able to rule out Einstein’s local realism or establish QM’s predicted, non-local entanglement as an actual phenomenon.

Google “loopholes in Bell test experiments” and you will see. All experiments to date suffer from problems, generally either insufficient detection rates or cross-talk problems. The Bell’s Theorem inequalities, correctly accounting for real experiments, are not violated.

The best Bell’s Theorem experiment to date – I think it’s Ansmann’s – seems almost close to establishing “spooky action at a distance” but is compatible with and actually accurate enough to be in support of local realism.

I understand Bell’s Inequalilty and the modified inequality that takes account of detection inefficiencies. Surely potential detection inefficiencies affect the prediction of QM. However, I have never seem a modified QM prediction that also takes account of detection inefficiencies. Can you direct me to any such accounts?

They show up a lot in the experimental papers that actually verify the Bell inequalities. Errors are a pretty standard problem (they’re the problem), and dealing with them is such old hat that it generally gets glossed over.
Paul Kwiat and Anton Zeilinger have both done a number of (fairly dense) papers on this sort of thing.

It’s already been mentioned, but it’s worth restating, that a violation of Bell’s Inequality does not disprove the possibility of hidden variables. Bell himself was a fan of Bohm’s theory, which is a hidden variables theory.

The theorem states that there is a difference between the predictions of theories based on local hidden variables and those of QM. The key word here is ‘local’. Violations of Bell’s Inequality suggest that at least one of the assumptions he made in proving his result (which he took from the Einstein-Podolsky-Rosen argument) must be false: either nature is non-local or hidden variables do not exist (or both).

But in the EPR argument, the orthodox interpretation of QM (non-real, no hidden variables) was shown to require non-locality to account for QM’s predicted correlations between measurements separated by a sufficient distance. Einstein simply would not countenance the idea that nature could be non-local, that there was no ‘spooky action at a distance’, and so he concluded that QM was incomplete because it had no other way of accounting for its predicted correlations.

So even if Bell’s Theorm is taken (wrongly) as demonstrating that there are no hidden variables (non-real), the EPR argument is that nature must still be non-local. So, what violations of Bell’s Inequality suggest is that nature is non-local.

I believe that since the above posts were made, there has been significant progress in closing the loopholes that would still permit a local hidden variable theory. It’s still possible nature does employ hidden variables, but if she does, she does so non-locally.

I am a Learner with a capital ‘L’ when it comes to this area so forgive me if my question/comment is ridiculous, but here goes.
Is one possible answer to this anomaly (and various others I’ve heard of) that the whole cosmos down to its smallest particle exists within another ‘reality’ that has its own and different set of laws governing the behavior of its constituent parts? (and that includes our cosmos which is within the greater reality). If that were so then when we get down to the quantum level we are looking at the interface between the physical reality that we know and experience and this other greater reality. This ‘other reality’ can only be discerned when we come across these ‘odd behavior’ incidents. An oversimplification perhaps, but the existence of one ‘other and greater reality’ would maybe do away with the need to imagine the multiplicity of realities that have been posited by some leaders in the field.