The trouble with shadow photons

Chapter 7 of David Deutsch's "The Fabric of Reality" should be considered required reading for anyone who wants to understand the scientific process. Unfortunately, it is embedded in a book that is full of some crazy ideas, at least one of which is wrong even by Deutsch's own standards. This one mistake, which Deutsch makes very early on, infects the rest of his reasoning like a virus. This is very unfortunate because I think this is at least in part to blame for the fact that the book as a whole has not gotten the attention it deserves.

It makes a very interesting intellectual exercise to try to uncover Deutsch's mistake on your own. You don't need to be a physicist to find it. Here are two hints to get you started: the mistake is in chapter 2, and everything you need to know in order to figure it out is in this paper, which is written to be accessibly to a lay audience.

So at the risk of sounding like a broken record, get ye over to Amazon and buy a copy of Deutsch's book. Read chapter 7. Then read chapter 2 and try to solve this puzzle for yourself before you read any further. I promise you it will be time well spent. This post is not just about pointing a finger at David Deutsch and saying "ha! ha!" I want to make a much deeper point about how challenging it can be to figure out what is "true". It will help if you have the visceral experience of reading Deutsch's argument and trying to figure out for yourself what is wrong with it even if (perhaps especially if) you don't succeed.

Last warning. Spoiler alert.

Deutsch's mistake is in this paragraph, which in my copy is on page 43:

"Could it be that the photon splits into fragments which, after passing through the slits, change course and recombine? We can rule out that possibility too. If, again, we fire one photon through the apparatus, but use four detectors, one at each slit, then at most one of them ever registers anything. Since in such an experiment we never observe two of the detectors going off at once, we can tell that the entities that they detect are not splitting up."

In my "Quantum Mysteries Disentangled" paper I use the metaphor of a magic trick to describe how popular accounts of QM spread confusion. Most people think that magic is all about gimmicks and skillful sleight-of-hand, but the heart of a good magic trick is a narrative that seems plausible but in fact does not reflect the underlying reality. The elements of this narrative can be quite subtle. I take the Queen of Spades and I turn it face down and I put it, uh, over here. But in fact the card that I'm putting over here is not the queen of spades at all; I've already switched it out for a different card. Much later when the true identity of the card is revealed, you will swear on a stack of Bibles that that card was the queen of spades. You saw it with your own eyes. And you were watching it like a hawk the whole time. Except that by the time you started watching like a hawk the trick was already done. It happened during that little moment of confusion, when I lost my focus and couldn't quite remember what I was supposed to do next. In fact, I didn't lose my focus. Every move was scripted in minute detail. I say "uh" to make it look like I've lost my train of thought. That makes you take your guard down because if I've lost my train of thought then I'm not doing the trick. I look around trying to decide what I'm supposed to do with the card, and your gaze follows mine because deep down in your brain there are instincts that say when a member of your tribe is focusing their attention somewhere you should check it out too because they might be looking at a saber-tooth tiger. In that instant, I absent-mindedly put the card briefly back on the top of the deck before I put it over here. And in that instant I make the switch. Except that I don't even have to make a switch because in fact that card that I put on top of the deck was not the queen of spades either, it was two cards, with the queen of spades on the bottom.

Deutsch's mistake is a sin of omission: he fails to mention the crucial fact that when you do this experiment with the four detectors the interference goes away. This is a devastating fact for Deutsch's theory of shadow photons, but before I explain why there is another important thing to take note of, namely, that at this point in the argument Deutsch has not yet introduced the theory of shadow photons. This is what makes it so easy to miss that Deutsch has omitted an important fact.

The fact that adding detectors destroys interference is fatal for the theory of shadow photons. To understand why, we need to recap exactly what it is that the theory of shadow photons says: something influences the behavior of "real" (what Deutsch calls "tangible") photons to produce interference. Whatever that "something" is behaves just like "real" photons, except that it is "detectable only indirectly through ... interference effects ...". So we might as well call them (shadow) photons. Since shadow photons exist (because they have observable effects), it must be the case that entire shadow universes exist spanning an incomprehensible gamut of possible configurations, including some (actually incomprehensibly many) where David Deutsch does not exist, and therefore -- and this is crucial -- some in which the detectors on the slits do not exist.

This is the odd thing about shadow photons: they behave just like photons that exist in parallel universes except that their movements correspond exactly to the movements of photons in this universe. There are only two possibilities. One is that our universe is somehow privileged, and what we do here produces miraculous effects in the myriad shadow universes. If we decide to cover our slits then the photons in all the shadow universes are blocked, even those in shadow universes where the slits are still open. Imagine how puzzled those poor scientists must be trying to figure out why light behaves as it does. There is no correspondence between how light behaves in their universe and the physical configuration of their universe. Sometimes light passes through blocked slits, and sometimes it fails to pass through open slits. From this behavior they could likewise infer the presence of what is to them a shadow universe (which to us is the real universe) that governs the behavior of photons in their universe, but that just begs the question of why we happen to inhabit the one privileged universe whose macroscopic configuration governs the behavior of light in all of the shadow universes.

The other possibility is to postulate a new rule for shadow photons: only those shadow photons that come from shadow universes whose macroscopic configuration is the same as the "real" universe are allowed to influence "real" photons. But if we postulate that then it immediately follows that shadow universes with different macroscopic configurations have no influence on our universe whatsoever and therefore, by Deutsch's own criteria, do not exist.

One way or another, there is no escaping the fact that our universe is somehow special.

Now, the interesting this about this to me is not so much that Deutsch was wrong per se, but the process by which he came to be wrong. Unlike a magician, I'm pretty sure Deutsch did not set out intentionally to deceive. I'm also pretty sure that Deutsch didn't get it wrong because he was simply being stupid. To the contrary, I think Deutsch is probably brilliant, a lot smarter than me. For one thing, he's actually a physicist, which I'm pretty sure even now still makes an effective filter for idiocy. So what went wrong?

I have no way of knowing for sure, of course, but here's my best guess: I think Deutsch actually managed to fool himself. Deutch's book is, notwithstanding that much of it is actually wrong, a model of clear thinking. To find the flaw in his reasoning is not easy, in no small measure because of subtle details of how he presents his argument. This is why I urged you to actually do the exercise yourself.

The real point of this essay is not to harsh on David Deutsch, but to show that even very smart people, even scientists (sometimes especially scientists) can and do get things wrong, and sorting out the resulting mess is not always easy. This is one of many reasons why both scientists and Scientists ought always to maintain a healthy measure of humility.

UPDATE:

Here's what David Deutsch had to say about this (via email -- I'm pretty sure he hasn't actually read this post):

RG: I know you probably hear this a lot, but I believe I have found a serious flaw in the theory of shadow photons. In a nutshell, the rules of motion for shadow photons are governed by the macroscopic configuration of *our* universe. If a slit is open in *our* universe, both tangible and shadow photons pass through. If a slit is blocked in *our* universe, both tangible and shadow photons are blocked. It seems to me there are only two possibilities: either the macroscopic configuration of *our* universe governs the movement of shadow photons (in which case shadow-scientists must be mightily puzzled why their tangible photons sometimes pass through solid objects, and sometimes fail to pass through open slits), or we have to postulate that only shadow photons from universes whose macroscopic configuration matches our own can interact with our tangible photons.

DD: The latter is, to a good approximation, the case.

RG: But if we postulate this, then shadow photons from universes whose macroscopic configurations do not match ours cannot interact with our universe in any way, and therefore by your own criteria do not exist. Or have I missed something?

DD: You're claiming there's an inadequacy in the criterion for existing that I presented, not a flaw in the 'theory of shadow photons' (which is aka quantum theory). However, that criterion was not intended as a criterion of what *doesn't* exist. If it were used in that way, then we would have to classify all the photons that have left the sun, and passed the Earth, and are never going to strike anything in the future, as being nonexistent.

My response:

1. I find it disingenuous to claim that the theory of shadow photons is "a.k.a. quantum theory." The theory of shadow photons is in fact a.k.a. Hugh Everett's relative state formulation of quantum mechanics, which was later renamed the many-worlds interpretation by Bryce DeWitt. To be sure it is taken seriously by a great many people, but to say without qualification that it is quantum theory is just flat-out wrong. I appreciate Deutsch's intellectual honesty in essentially admitting that my criticism is valid, but I'm puzzled by how he can rail against arbitrary complexity in scientific explanations on the one hand, and then accept as essentially inarguable the proposition that parallel universes are causally connected on a microscopic level by virtue of their macroscopic configurations. It seems plain to me that such an "explanation" of QM gets you exactly nowhere. But I suppose we'll just have to agree to disagree about that.

2. John Cramer's transactional interpretation of quantum mechanics actually does consider photons that never interact with anything to be non-existent, so this is not as outlandish a notion as Deutsch implies.

UPDATE 2:

Deutsch just referred me to this paper which is the more formal formulation of his multiple-worlds theory. I must confess that on a cursory read it seems to be a compelling argument. So I may have to rethink this whole thing.

Just for the record, my (current) disagreement with Deutsch is not huge. It's clear from the structure of QM that the proposition that there is only one (classical) universe is untenable. But that still leaves you with the choice of an infinite number of universes (Deutsch's position) or zero (mine). It has always seemed to me that zero was the more parsimonious choice, but I'm now beginning to have some doubts.