Saturday, June 08, 2019

How do we know that quantum randomness is really random?

Since the dawn of quantum physics, the Born rule has been the cause of much consternation and gnashing of dentition, with Einstein famously complaining that God doesn't play dice.    Was Einstein right?  Is the apparent randomness of quantum measurements an illusion?  Are the results of quantum measurements actually deterministic but dependent on some hidden state that we simply don't have access to?  In other words, is the apparent randomness a reflection of a fundamental truth about objective reality, or simply a reflection of our ignorance?  And how can we possibly know for sure?  After all, ignorance, by its very nature, does not yield readily to introspection.

For the purposes of this discussion let's consider an idealized quantum experiment that has only two outcomes.  You can think of this as measuring the spin of an electron or the polarization of a photon.  Again, the details don't matter.  All that matters is that there are two possible outcomes (let's call them A and B).  Let's further suppose just for the sake of simplicity that both outcomes are equally likely.

So we do a bunch of experiments and collect a bunch of data.  This data is in the form of a sequence of A's and B's, each corresponding to the outcome of one instance of the experiment.  We analyze this sequence and it looks random.  We apply a bunch of statistical tests and they call come back and say, yep, this is a random sequence.  We wrack our brains to try to come up with a way to predict the outcome of the next experiment with odds better than chance and we fail.  Does that prove that this sequence is in fact random?

No, of course not.  We can trivially reproduce the exact same sequence in a purely deterministic way simply by playing back the record of the previous outcomes!  So how can we know that the sequence wasn't generated this way the first time around?  How do we know that there isn't a deterministic process out there in the universe somewhere that has pre-ordained the outcome of every quantum measurement we ever make?

It would seem that the fact that any sequence of experimental outcomes could be generated by playing back a record of them shows that we can never be sure that it wasn't actually done that way.  But this is wrong, and in fact it is easy to see exactly how and why it is wrong.  You might want to stop here and see if you can figure it out on your own.

Here's a clue: if the sequence of experimental outcomes was the result of simply replaying a record, or indeed of any kind of deterministic computational process (of which replaying a record is just one trivial example), we should be able to find evidence of that somewhere.  In order to make a record we have to store information somewhere.  In order to make a computational process we have to make a computer.  If that information was stored in any kind of straightforward way, and in particular (no pun intended) if that information is stored in any kind of physical artifact made of atoms then we should be able to find it.  Or at least we should be able to find some evidence that it exists.

But we can't.  And if you think about it, it is absolutely impossible for such an artifact to exist.  Why? Because it would have to store the outcome not just for the experiments that we actually do, but for any experiment that we could possibly do, and it would have to store those outcomes for every particle in the universe on which we might choose to perform an experiment including the particles that make up the artifact that is supposedly storing all this information!

Now, this is not yet an ironclad proof because there is one remaining possibility.  We don't actually have to store a separate record of the results of our experiments in order to be able to reproduce the same sequence of results.  It is enough simply to hang on to the particles themselves.  Once we have made a measurement on a particle, if we make the same measurement again we will get the same result.  So it's possible that the information that determines the results of experiments performed on particles is stored in the particles themselves.

The usual way to dispense with this possibility is to invoke Bell's theorem and to point out that it rules out local hidden variables.  But there's a more elementary (and, I think, more compelling) argument.

It is true that if we do the same experiment on a single particle twice in a row we will get the same result.  But nothing constrains us from doing different experiments on a single particle.  We could, for example, measure the position of a particle, and then its momentum, and then its position again, and then its momentum again.  If we do this, every result will be (apparently) random, completely disconnected from anything that has gone before.  And (and this is key) we can do this forever.  We can perform an infinite (well, OK, unbounded) number of measurements on one particle.  For the results to be deterministic, the state of the particle would have to store an infinite (and this time I really do mean infinite) amount of information.

Well, how do we know that this is not in fact the case?  Bohmian mechanics, for example, is a theory of exactly this sort.  In Bohmian mechanics, particles have positions, and (it turns out) all of the potentially infinite information that can be read out (eventually) via quantum measurements is encoded in the initial position of the particle.  This is possible because the position of the particle is metaphysically exact, represented by an actual real number with an infinite number of digits in its expansion and hence can contain an infinite amount of information.

How do we know that this is not what is actually happening?

Well, we don't.  Bohmian mechanics reproduces the predictions of quantum mechanics exactly so there is no way to settle the question experimentally.  There are nonetheless three good reasons to reject Bohm as an adequate explanation of physical reality.

First, the way that Bohm handles spin is really weird, bordering on the perverse.  Again for those who don't know, spin is a property of certain particles (mainly electrons) that can be measured and always produces one of two results (usually called "up" and "down" even though these don't actually have any physical significance).  In Bohmian mechanics, the only metaphysically real property that a particle has is position (and hence also velocity, which is just the time derivative of the position as in classical mechanics).  Spin is not part of the metaphysically real state of a particle.  When you think you're measuring spin, you're actually measuring the particle's position (because that's all there is) but the wave function (the "pilot wave", the thing that's pushing the particle around) conspires to move the particle through spin-measurement apparatus in just such a way that it looks as if the particle has spin, even though it really doesn't.  Bohmian mechanics is quite literally a cosmic conspiracy theory.

Second, Bohmian mechanics is causally non-local.  When you do an EPR-type experiment Bohm says that the underlying metaphysical reality is different depending on the order in which you perform the two measurements.  But according to relativity, the order of space-like separated events is not well defined.  So in order to extract an unambiguous description of physical reality from Bohm you have to arbitrarily assign an order to space-like separated events (the technical term for this is choosing a foliation).  There is no way to tell which foliation is correct (if there were then you could experimentally falsify relativity) so the choice is arbitrary.  But (and this is the crucial point) the fact that it is arbitrary completely undermines the whole point of adopting Bohmian mechanics to begin with, which was to provide a complete description of physical reality that was compatible with classical intuition.  Instead of one description of reality you have an arbitrary number of them, one for each possible foliation, and there's no way to tell which one is actually correct.

Third, although Bohm hangs his hat entirely on the physical reality of particle positions, it is fundamentally impossible to know what the position of a particle actually is.  Remember, when you measure the spin of a particle, on Bohmian mechanics you are not really measuring spin, you are really measuring position (because that's all there is).  Well, it turns out that when you measure position you aren't really measuring position either.  The only kind of measurement you can make is one that tells you whether or not the particle was inside or outside a particular (no pun intended) finite region of space.  The actual position of a particle cannot be measured.  So the one thing that Bohm advances as a description of physical reality turns out to be the operational equivalent of an invisible pink unicorn (IPU) — a thing that is posited to exist but which, by its very nature, can never be measured.

There is an even simpler argument to demonstrate the non-measurability of Bohmian positions: a measurement can only ever produce a finite amount of information, but the information encoded in the particle's actual metaphysical position is necessarily infinite (because it has to encode the results of all possible future measurements).  So the results of position measurements must contain errors.

And this is the crux of the matter: it just turns out as a matter of physical fact that a single particle can produce what appears to be an unbounded amount of information.  There are only two possibilities: either that information is generated on the fly ex nihilo, (that is, it's "really random") or it is stored somewhere.  But no one has been able to identify any possible repository in the physical world where that information could be stored.  In fact, QM fundamentally depends on this not being possible!  Interference effects only manifest themselves if there is no possible way to distinguish two different states of a system.  But to contain information, a system must have distinguishable states -- that's what information means!  To be compatible with the data, a hypothetical repository of that information must necessarily be an IPU.  Any theory where the existence of such a repository was experimentally demonstrable would not be compatible with QM.

Thursday, June 06, 2019

Quantum transitions take time. This is not news.

A number of people have asked me to weigh in on this story in Quanta Magazine (based on this paper [PDF version] and also reported in this press release from Yale, and several other popular outlets.)

Here's how Quanta breathlessly reported the result:
When quantum mechanics was first developed a century ago as a theory for understanding the atomic-scale world, one of its key concepts was so radical, bold and counter-intuitive that it passed into popular language: the “quantum leap.” Purists might object that the common habit of applying this term to a big change misses the point that jumps between two quantum states are typically tiny, which is precisely why they weren’t noticed sooner. But the real point is that they’re sudden. So sudden, in fact, that many of the pioneers of quantum mechanics assumed they were instantaneous. 
A new experiment shows that they aren’t.
This is mostly hype.  While it is true that in the very early days of quantum mechanics some researchers (notably Niels Bohr) thought that quantum transitions were instantaneous, the fact that they aren't has been known for decades.  What is new here is that this is the first time that this fact has been demonstrated experimentally.  I don't want to detract from the technical accomplishment here in any way, it's a truly impressive experiment.  But it's not the kind of conceptual breakthrough that the Quanta story implies.  It's a totally expected result.

It is natural to conclude from the fact that energy states are quantized that the transition between them must happen instantaneously.  Consider a system that transitions from energy state 0 to an adjacent energy state 1 (in some suitable units). It can't do it via a smooth transition between intermediate energy levels because these are physically impossible (that the whole point of quantum mechanics).  So if a system is going to transition from 0 to 1 without occupying any energy state in between, the transition must be instantaneous, right?

Wrong.  There is a different kind of "smooth" transition that a system can make between the 0 and 1 states, and that is via a superposition of the two states.  Just as a particle can be in two different locations at the same time, it can be in two different energy states at the same time.  To go smoothly from 0 to 1, the system transitions through a series of superpositions of both states, i.e. it starts out entirely in state 0, and then transitions smoothly to being mostly in state 0 and a little bit in state 1, to being half in each state, to being mostly in 1 and a little bit in 0, to being entirely in 1.  This has been known for decades, and is predicted by the math.  You can even predict how fast the transition happens.  For most common physical processes, like an atom absorbing or emitting a photon, the transition is really fast.  But it's not instantaneous.

The tricky part is not figuring out that quantum transitions take time (well, OK, figuring it out is tricky too, but it's easy once you know how) but designing an experiment that demonstrates that the theory is correct.  This is because any straightforward measurement of the energy of the system will always produce a result that shows the system is in one state or the other.  The existence of superpositions can only be demonstrated indirectly, usually through interference effects.  So to demonstrate the non-instantaneous nature of a quantum transition you have to do two things: first, you need to actually catch a system during a (typically very fast) transition and second, you need to come up with a way of getting the system to interfere with itself (or producing some other indirect effect that would not occur but for the existence of a superposition).  That's what Minev et al. did.

The way they did it is really cool, but the advance here is an experimental one, not a theoretical one.  They used a superconductor to produce a macroscopic quantum system that behaved like an atom in that it had a small number of discrete energy levels that it could transition between.  Then they "tickled" this "atom" with microwaves and observed that the resulting response exhibited the kind of interference effects that would be expected if if were transitioning through superposition states.  It's very cool, and a very impressive technical achievement, but it is in no way unexpected or surprising.