The Trouble with Many Worlds

Ten years ago I wrote an essay entitled "The Trouble with Shadow Photons" describing a problem with the dramatic narrative of what is commonly called the "many-worlds" interpretation of quantum mechanics (but which was originally and IMHO more appropriately called the "relative state" interpretation) as presented by David Deutsch in his (otherwise excellent) book, "The Fabric of Reality."  At the end of that essay I noted in an update:

Deutsch just referred me to this paper which is the more formal formulation of his multiple-worlds theory. I must confess that on a cursory read it seems to be a compelling argument. So I may have to rethink this whole thing.

That paper is entitled "The Structure of the Multiverse" and its abstract is delightfully succinct.  I quote it here in its entirety:

The structure of the multiverse is determined by information flow.

Those of you who have been following my quantum adventures know that I am a big fan of information theory, so I was well primed to resonate with Deutsch's theory.  And I did resonate with it (and still do).  Deutsch's argument was compelling (and still is).  Nonetheless, I never wrote a followup for two reasons.  First, something was still bothering me about the argument, though I couldn't really put my finger on it.  Yes, Deutsch's argument was compelling, but on the other hand, so was my argument (at least to me).  The difference seemed to me (as many things in QM interpretations do) a matter of taste, so it seemed pointless to elaborate.  And second, I didn't think anyone reading this blog would really care.  So I tabled it.

But last May the comment thread in the original post was awakened from its slumber by a fellow named Elliot Temple.  The subsequent exchange led me to this paper, of which I was previously unaware.  Here's the abstract, again, in its entirety:

The probabilistic predictions of quantum theory are conventionally obtained from a special probabilistic axiom.  But that is unnecessary because all the practical consequences of such predictions follow from the remaining, non-probabilistic, axioms of quantum theory, together with the non-probabilistic part of classical decision theory.

The "special probabilistic axiom" to which Deutsch refers is called the Born rule (named after Max Born).  The "remaining, non-probabilistic axioms of quantum theory" comprises mainly the Schrödinger equation.  (To condense things a bit I'll occsaionally refer to these as the BR and the SE.)

The process of applying quantum mechanics to real-world situations consists of two steps: first you solve the SE.  The result is something called a "wave function".  Then you apply the BR to the wave function and what pops out is a set of probabilities for various possible results of the experiment you're doing.  Following this procedure yields astonishingly accurate results: no experiment has ever been done whose outcome is at odds with its predictions.  The details don't matter.  What matters is: there's this procedure.  It yields incredibly accurate predictions.  It consists of two parts.  One part is deterministic, the other part isn't.

This naturally raises the question of why this procedure works as well as it does.  In particular, why does the procedure have two parts?  And why does it only yield probabilities?  Answering these questions is the business of "interpretations" of quantum mechanics.  Wikipedia lists almost twenty of these.  The fact that after nearly 100 years no consensus has emerged as to which one is correct gives you some idea of the thorniness of this problem.

So the paper that Elliot referred me to was potentially a Big Deal.  It is hard to overstate the magnitude of the breakthrough this would be.  It would show that there are not in fact two disparate parts to the theory, there is only one: the SE.  Such a unification would be of the same order of magnitude as the discovery of relativity.  It would be headline news.  David Deutsch would be a Nobel Laureate, on a par with Newton and Einstein.  But the fact that there is still an active debate over the issue shows that Deutsch's claim has not been universally accepted.  So there would seem to be only two possibilities: either Deutsch is wrong, or he's right and the rest of the physics community has failed to recognize it.

Normally when a claim of a major result like this fails to be recognized by the community it's because the claim is wrong.  In fact, more than 99% of the time it's because the claimant is a crackpot.  But Deutsch is no crackpot.  He's a foundational figure in quantum computing.  He discovered the first quantum algorithm.  Even if he got something wrong he very likely got it wrong in a very interesting way.

So I decided to do a deep dive into this.  It led me down quite the little rabbit hole.  There are a number of published critiques of Deutsch's work, and counter-critiques critiquing the critiques, and counter-counter-critiques.  They're all quite technical.  It took me a couple of months of steady effort to sort it all out, and that only with the kind of help of a couple of people who understand all this stuff much better than I do.  (Many thanks to Tim Maudlin, David Wallace, and especially the patient, knowledgeable, and splendidly-pseudonymed /u/ididnoteatyourcat on Reddit.)

In the rest of this post I'm going to try to describe the result of going down that rabbit hole in a way that is accessible to what I think is the majority of the audience of this blog.  The TL;DR is that Deutsch's argument depends on at least one assumption that is open to legitimate doubt.  Figuring out what that assumption is isn't easy, and whether or not the assumption is actually untrue is arguable.  That's the reason that Deutsch hasn't won his Nobel yet.

I have to start with a review of the rhetoric of the many-worlds interpretation of quantum mechanics (MWI).  The rhetoric says that when you do a quantum measurement it is simply not the case that it has a single outcome.  Instead, what happens is that the universe "splits" into multiple parts when a measurement is performed, and so all of the possible outcomes of an experiment actually happen as a matter of physical fact.  The reason you only perceive a single outcome is that you yourself split into multiple copies.  Each copy of you perceives a single outcome, but the sum total of all the "you's" that have been created collectively perceive all the possible outcomes.

I used the word "rhetoric" above because, as we shall see, there is a disconnect between what I have just written and the math.  To be fair to Deutsch, his rhetoric is different from what I have written above, and it more closely matches the math.  Instead of "splitting", on Deutsch's view the universe "peels apart" (that's my terminology) in "waves of differentiation" (that is Deutsch's terminology) rather than "splitting" (that is everyone else's terminology) but this is a detail.  The point is that at the end of a process that involves you doing a quantum measurement with N possible outcomes, there are, again in point of actual physical fact, N "copies" of you (Deutsch uses the word "doppelgänger").

Again, to be fair to Deutsch, he acknowledges that this is not quite correct:

Universes, histories, particles and their instances are not referred to by quantum theory at all – any more than are planets, and human beings and their lives and loves. Those are all approximate, emergent phenomena in the multiverse.  [The Beginning of Infinity, p292, emphasis added.]

All of the difficulty, it will turn out, hinges on the fidelity of the approximation.  But let us ignore this for now and look at Deutsch's argument.

Deutsch attempts to capture the idea of probability in a deterministic theory using game theory, that is, by looking at how a rational agent should act, applying a few reasonable-looking assumptions about the utility function, and showing that a rational agent operating under the MWI would act exactly as if they were using the Born rule.  The argument is long and technical, but it can be summarized very simply.

[Note to nit-pickers: this simplified argument is in fact a straw man because it is based on the assumption that branch counting is a legitimate rational strategy, which is actually false on the Deutsch-Wallace view.  But since the conclusion I am going to reach is the same as Deutsch's I consider this legitimate rhetorical and literary license because the target audience here is mainly non-technical.]

For simplicity, let's consider only the case of doing an experiment with two possible outcomes (let's call them A and B).  The game-theoretical setup is this: you are going to place a bet on either A or B and then do the experiment.  If the outcome matches your choice, you win $1, otherwise you lose $1.

If the experiment is set up in such a way that the quantum-mechanical odds of each outcome are the same (i.e. 50-50) then there is no conflict between the orthodox Born-rule-based approach and the MWI: in both cases, the agent has no reason to prefer betting on one outcome over the other.  The only difference is the rationale that each agent would offer: one would say, "The Born rule says the odds are even so I don't care which I choose" and the other would say, "I am going to split into two and one of me is going to experience one outcome (and win $1) and the other of me is going to experience the other outcome (and lose $1), and that will be situation no matter whether I choose A or B, so I don't care which I choose."

[Aside: Deutsch goes through a great deal more complicated argument to prove this result because it is based on an assumption that Deutsch rejects.  In fact, he goes on from there to put in a great deal more effort to extend this result to an experiment with N possible outcomes, all of which have equal probabilities under the Born rule.  He has to do this because my argument is based on a tacit assumption that Deutsch rejects.  We'll get to that.  My goal at this point is not to reproduce Deutsch's reasoning, only to convince you that this intermediate result is plausibly true.]

Now consider a case where the odds are not even.  Let's arrange for the probabilities to be 2:1 in favor of A (i.e. A happens 2/3 of the time, B happens 1/3 of the time, according to the Born rule).  Now we have a disconnect between the two world-views.  The Bornian would obviously choose A.  But what possible reason could the many-worlder have for doing the same?  After all, the situation is unchanged from before: again the many-worlder is going to split into two (because there are still only two possible outcomes).  What possible basis could they have for preferring one outcome over the other that doesn't assume the Born rule and hence beg the question?

Deutsch's argument is based on an assumption called branching indifference.  Deutsch himself did not make this explicit in his original paper, it was clarified by David Wallace in a follow-up paper.  Branching indifference says that a rational agent doesn't care about branching per se.  In other words, if an agent does a quantum experiment that doesn't have a wager associated with it, then the agent has no reason to care whether or not the experiment is performed or not.

The reasoning then proceeds as follows: suppose that the many-worlder who ends up on the A branch does a follow-up experiment with two outcomes and even odds, but without placing a bet.  Now there are three copies of him, two of which have won $1 and one of which has lost $1.  But (and this is the crucial point) all of these copies are now on branches that have equal probabilities.  Because of branch indifference, this situation is effectively equivalent to one where there was a single experiment with three outcomes, each with equal probability, but two of which result in winning $1, and where the agent had the opportunity to place the bet on both winning branches.

So that sounds like a reasonable argument.  In fact, it is a correct argument, i.e. the conclusions really do follow from the premises.

But are the premises reasonable?  Well, many many-worlders think so.  But I don't.  In particular, I cast a very jaundiced eye on branching indifference.  There are two reasons for this.  But first, let's look at Wallace's argument for why branching indifference is reasonable:

Solution continuity and branching indifference — and indeed problem continuity — can be understood in the same way, in terms of the limitations of any physically realisable agent. Any discontinuous preference order would require an agent to make arbitrarily precise distinctions between different acts, something which is not physically possible. Any preference order which could not be extended to allow for arbitrarily small changes in the acts being considered would have the same requirement. And a preference order which is not indifferent to branching per se would in practice be impossible to act on: branching is uncontrollable and ever-present in an Everettian universe.

If that didn't make sense to you, don't worry, I'll explain it.  But first I want to take a brief diversion.  Trust me, I'll come back to this.

Remember how I said earlier that my simplified argument for Deutsch's conclusion was based on a premise that Deutsch would reject?  That premise is called branch counting.  It is the idea that the number of copies of me that exist matters.  This seems like an odd premise to dispute.  How could it possibly not matter if there is one of me winning $1 or a million of me each winning $1?  The latter situation might not have a utility that is a million times higher than the former, but if I'm supposed to care about "copies of me" at all, how can it not matter how many there are?

Here is Wallace's answer:

Why it is irrational: The first thing to note about branch counting is that it can’t actually be motivated or even defined given the structure of quantum mechanics. There is no such thing as “branch count”: as I noted earlier, the branching structure emergent from unitary quantum mechanics does not provide us with a well-defined notion of how many branches there are.

Wait, what???  There is no "well defined notion of how many branches there are?"

No, there isn't.  Wallace reiterates this over and over:

...the precise fineness of the grain of the decomposition is underspecified 

There is no “real” branching structure beyond a certain fineness of grain... 

...agents branch all the time (trillions of times per second at least, though really any count is arbitrary) 

...in the actual physics there is no such thing as a well-defined branch number

Remember how earlier I told you that there was a disconnect between the rhetoric and the math?  That the idea of "splitting" or "peeling apart" or whatever you want to call it was an approximation?  Well, this is where the rubber meets the road on that approximation.  Branching indifference is necessary because branching is not a well-defined concept.

So what about the rhetoric of MWI, that when you do an experiment with N possible outcomes that you split/peel-apart/whatever-you-want-to-call-it into N copies of yourself?  That is an approximation to the truth, but like classical reality itself, it is not the truth.  The actual truth is much more complex and subtle, and it hinges on what the word "you" means.

If by "you" you mean your body, which is to say, all the atoms that make up your arms and legs and eyes and brain etc. then it's true that there is no such thing as a well-defined branch count.  This is because every atom — indeed, every electron and every other sub-atomic particle — in your body is constantly "splitting" by virtue of its interactions with other nearby particles, including photons that are emitted by the sun and your smart phone and all the other objects that surround you.  These "splits" propagate out at the speed of light and create what Deutsch calls "waves of differentiation", what I call the "peeling apart" of different "worlds".  (If you are a regular reader you will have heard me refer to this phenomenon as creating "large systems of mutually entangled particles".  Same thing.)  This process is a continuous one.  There is never a well-defined "point in time" where the entire universe splits into two, and no point in time where you (meaning your body) splits into two.  There is a constant and continuous process of "peeling apart".  Actually many, many (many!) peelings-apart, all of which are happening continuously.  To call it mind-boggling would be quite the understatement.

On the other hand, if by "you" you mean "the entity that has subjective experiences and makes decisions based on those experiences" then things are much less clear.  I don't know about you, but my subjective experience is that there is exactly one of me at all times.  I consider this aspect of my subjective experience to be an essential component of what it means to be me.  I might even go so far as to say that my subjective experience of being a single unified whole defines what it is to be "me".  So the only way that there could be a "copy of me" is if there is another entity that has a subjective experience that is bound to the same past as my own, but whose present subjective experience is somehow different from my own e.g. my experiment came out A and theirs came out B.  An entity whose subjective experience is indistinguishable from my own isn't a copy of me, it's me.

The mathematical account of universes "peeling apart" has nothing to say about when the peeling process has progressed far enough to be considered a fully-fledged universe in its own right and so it has nothing to say about when I have "peeled apart" sufficiently to be considered a copy.  That is why branch count is not a coherent concept.

And yet, if I am going to apply the notion of branching to myself (which is to say, to the entity having the subjective experience of being a coherent and unified whole) then branch count must be a coherent concept.  It might not be possible to know the branch count, but at any point in time whatever underlying physical processes are really going on,  it has to either qualify as me branching or not.  There is no middle ground.

So we are faced with this stark choice: we can either believe the math, or we can believe our subjective experiences, but we can't do both, at least not at the same time.  We can take a "God's eye view" and look at the universal wave function, or we can take a "mortal's-eye view" and see our unified subjective experience as real.  But we can't do both simultaneously.  It's like a Necker cube.  You can see it one way or the other, but not both at the same time.

Interestingly, this is all predicted by the math!  In fact, the math tells us why there is this dichotomy.  Subjective experience is necessarily classical because it requires copying information.  In order to be conscious, you have to be conscious of something.  In order to make decisions, you have to obtain information about your environment and take actions that affect your environment.  All of these things require copying information into and out of your brain.  But quantum information cannot be copied.  Only classical information can be copied.  And the only way to create copyable classical information out of a quantum system is to ignore part of the quantum system.  Classical behavior emerges from quantum systems (mathematically) when you trace over parts of the system.  Specifically, it emerges when you consider a subset of an entangled system in isolation from the rest of the system.  When you do that, the mathematical description of the system switches from being a pure state to being a mixed state.  Nothing physical has changed.  It's purely a question of the point of view you choose to take.  You can either look at the whole system (in which case you see quantum behavior) or you can look at part of the system (in which case you see classical behavior) but you can't do both at the same time.

As a practical matter, in our day-to-day lives we have no choice but to "look" only at "part" of the system, because "the system" is the entire universe.  (In fact, it's an interesting puzzle how we can observe quantum behavior at all.  Every photon has to be emitted by, and hence be entangled with, something.  So why does the two-slit experiment work?)  We can take a "God's-eye view" only in the abstract.  We can never actually know the true state of the universe.  And, in fact, neither can God.

Classical reality is what you get when you slice-and-dice the wave function in a particular way.  It turns out that there is more than one way to do the slicing-and-dicing, and so if you take a God's-eye view you get more than one classical universe.  An arbitrary number, in fact, because the slicing-and-dicing is somewhat arbitrary.  (It is only "somewhat" arbitrary because there are only certain ways to do the slicing-and-dicing that yield coherent classical universes.  But even with that constraint there are an infinite number of possibilities, hence "no well-defined branch count".)  But the only way you can be you, the only way to become aware of your own existence, indeed the only way to become aware of anything, is to descend from Olympus, ignore parts of the wave function, and become classical.  That leaves open the question of which parts to ignore.  To me, the answer is obvious: I ignore all of it except the parts that measurably effect the "branch" that "I" am on.  To me, that is the only possible rational choice.