For months now I have been writing the phrase "when we get to quantum mechanics..." Well, today we finally get to quantum mechanics (QM). QM is a big topic. I will not be able to cover it all, or even put a significant dent in it, in one blog post. The goal of this post is just to give you an overview of the lay of the land, a clean-sheet introduction to the topic for non-technical people who know nothing about QM other than a few catch phrases like "wave-particle duality" and "spooky action at a distance." I am ultimately going to demystify it all for you, but first I want to get everyone on the same page about exactly what needs demystifying. The pedagogy of QM has improved since I first started studying it seriously back in the 1990s, but it's still in a pretty dismal state, with lots of old misconceptions and unjustified tacit assumptions floating around. So I'm going to just start from scratch here.
The first thing you need to know is that there are two different quantum theories: quantum mechanics (QM) and quantum field theory (QFT). These are not the same. QFT is QM extended to include special (but not general) relativity. Most of the time this doesn't matter, especially for philosophical discussions, and so many people, including me, will often say QM when they really mean QFT. But there are a couple of things that you should be aware of, starting with the fact that both of these theories exist. You should also know that the Standard Model of particle physics is a QFT theory. It has to be because the Standard Model includes light, the ultimate relativistic phenomenon, and if you don't think of light in relativistic terms you will go seriously astray.
The easiest way to go astray (and I'm jumping ahead here, but I really want to prepare you for this) is to think of photons as little point particles that move through space at the speed of light. It's very easy to fall back on a classical model that will tell you that, for example, if a photon is absorbed by a particular atom at a particular place at a particular time, then it must have been emitted by some other particular atom located at some other particular place at some other particular time, and that if you take the distance between those two places and divide by the difference in the two times the result will be 186,000 miles per second. That seems plausible, but it is completely, utterly, 100% wrong, as we shall eventually see. But if you want to read ahead, here is a three-part series [1] [2] [3] I wrote a while back that discusses this.
I cannot emphasize enough how hard it is to escape this way of thinking. We are non-relativistic creatures. All of our everyday experience is non-relativistic. The underlying truth plays havoc with our intuitions, and QM makes that problem much, much worse. I'll try to ease you into it as gently as I can.
I am going to begin by describing the Problem that led to the discovery of QM. There is some pedagogical peril in this because the historical route from the Problem to its eventual solution was a long and winding road, full of rhetorical and philosophical pitfalls, some of which have people stuck in them to this very day. So I'm going to give you advance warning of the biggest of these traps, which is something called the Measurement Problem (MP). The MP was not the original Problem that QM was invented to solve, it was a Problem that arose in the course of developing QM. There is disagreement over whether or not the MP has been solved or not. I believe it has, and the goal of this series of posts will be to explain to you what I believe to be the solution and why. Again, not to leave you in suspense, the solution (as I see it) has to do with something called entanglement. I believe that one of the reasons that the MP seems so intractable is that most of the standard introductory pedagogy for QM doesn't mention entanglement at all. If it is introduced to beginners at all, it is usually in the form of a post script, an additional extra-weird thing to add to the already-long list of (apparently) intractably weird things that distinguish QM from other physical theories. The truth (and this is uncontroversial) is that measurement and entanglement are inextricably linked. I believe (and this is still controversial) that entanglement provides a complete account of measurement, but for the moment this is neither here nor there. What matters is the reason that this is not more widely addressed when introducing people to QM: entanglement is a phenomenon that only arises in systems consisting of more than one particle, but to keep things simple most introductory examples deal only with single particles. And I'm going to do the same, but you should keep very firmly in the back of your mind that there are a lot of things that happen with single particles which are very special cases that apply only to single particles. The behavior of quantum systems with more than one particle (which is, of course, most systems) is qualitatively different from single-particle systems. In QM the whole is vastly (and literally!) greater than the mere sum of its parts.
QM (along with relativity -- the two were developed at roughly the same time) was invented to solve a couple of different problems that had arisen in classical mechanics towards the end of the nineteenth century. By that time Newton's laws and Maxwell's (well, Heaviside's really) equations had become well-established, but they made a few predictions that did not align with experiment. The most important of these (historically speaking) had to do with something called "black-body radiation" which I am not going to get into here. There are a zillion articles about it on the web if you want the details. The TL;DR is that physicists were tearing their hair out trying to come up with an explanation that matched the experimental results when Max Planck, in what he later described as "an act of desperation," pulled a mathematical model out of a (metaphorical) hat that assumed -- with no motivation whatsoever other than that it made the results come out right -- that energy could only be transferred from one system to another in fixed-sized chunks, which he called "quanta". Fast forward to 1905 and Albert Einstein used a riff on Planck's math to explain another then-mysterious observation called the photoelectric effect. (This is actually the work that won Einstein a Nobel Prize.) Again, I'm not going to go into the details. What matters is that to this point all of the unexplained observations had to do with light. That changed in 1924 when Louis de Broglie (pronounced, more or less, "du broy") predicted in his 1924 PhD thesis that electrons would also exhibit wave-like behavior. It is hard to appreciate today just how crazy that sounded at the time. The question of whether or not atoms were real had only recently been considered to be fully settled. (Again, it was Einstein who provided the definitive proof in his theory of Brownian motion.) The electron itself had only been discovered less than 30 years prior, by J.J. Thomson in 1897. It was beyond question that the electron was a particle. Like Planck before him, de Broglie basically just pulled the idea out of a hat that if a wave (light) could sometimes behave like a particle, maybe particles (electrons) could sometimes behave like waves. There was absolutely no evidence for this at the time. But in 1959 de Broglie was vindicated when Claus Jönsson demonstrated that electrons produced an interference pattern in a two-slit experiment. Since then thousands upon thousands of painstaking calculations and experiments have shown that QM/QFT is the single best scientific theory ever devised. Its predictions have been verified in some cases out to 13 significant figures, comparable to measuring the diameter of the earth with an error of less than a micron.
The upshot of all this work was a theory that is quite challenging to explain to a lay audience. It doesn't help that one of the most prominent people involved in developing the theory, Richard Feynman, famously quipped that "no one understands quantum mechanics." It's not true. Quantum mechanics is no harder to understand than relativity. But it is a lot harder to explain. The reason is that to understand relativity you really only need to wrap your brain around one unintuitive idea, namely, the fact that the speed of light is constant in all reference frames. Quantum mechanics comprises half a dozen such unintuitive notions: the measurement problem, the uncertainty principle, wave-particle duality, entanglement (a.k.a. "spooky action at a distance"), many-worlds, and probably a few others.
It also doesn't help that the standard pedagogy for QM is highly misleading, at least in terms of its philosophical implications. As mentioned earlier, nearly all introductions to QM focus on single-particle systems when in fact most of the philosophically interesting stuff only starts to happen with multiple-particle systems. In fact, even the use of the word "particle" here is misleading, as we will eventually see. So it's really hard to figure out where to start picking at this tangled mess.
I'm going take the following approach: first I'm going to just describe what the theory looks like, a high-level overview of the math. Don't worry, I'm not going to do any actual math. (We will eventually have to do a little math, but again, don't worry, it' won't be to hard. If you remember high-school algebra you'll be fine.) The goal here will be just to give you a high-level overview of what the theory looks like, and to introduce you to some of the terminology. The upshot of all that will be that QM has to do with waves, and that leads to something you have probably heard of, the Heisenberg uncertainty principle, which says that you cannot simultaneously know the position and velocity of a particle. Note that this fact bears some resemblance to the impossibility of measuring the one-way speed of light. This is not an accident. It is a reflection of a deep underlying truth. In both cases the impossibility of measuring or knowing these things is not a limit of our technology or out cleverness, it is a result of a profound fact: the "one-way speed of light" is not merely beyond our epistemological grasp, it is a physically meaningless concept. And it is the same with simultaneous position and momentum. The reason we can't know them is that they literally cannot exist at the same time, and the reason for this is that the things we call "particles" aren't actually particles. They are things that under certain circumstances act like particles. But most of the time they act like waves, and waves literally cannot have a well-defined position and momentum at the same time.
One final thing by way of introduction: relativity messes with our intuitions because it mainly manifests at speeds and accelerations that are vastly higher than our day-to-day experience. QM messes with our intuitions because it mainly manifests at sizes and temperatures that are much smaller than our day-to-day experiences. Relativity manifests when things are very fast or very heavy. QM manifests when things are very small or very cold. But a big difference between relativity and QM is that it is pretty easy to understand how our day-to-day non-relativistic world arises from the relativistic one. If you plug slow speeds into the relativistic equations what pops out are results that align with intuition. This is not true for QM. Our day-to-day world appears to be qualitatively different -- indeed, fundamentally incompatible -- from what quantum mechanics describes. Explaining how our day-to-day world -- what is called in the trade the "classical" world -- arises from QM is one of the biggest challenges ever faced by humans. This is the famous (or infamous) "measurement problem". We won't get to that until later, but I wanted to warn you that this demon is lurking, and to promise that we will eventually face and slay him.
So let me start by pointing out the features that QM and classical mechanics share in common. Both of them work by taking a description of a state of the world and projecting that state forward (or backward) in time. The math that does this projection is called the dynamics of the theory. In classical mechanics the dynamics are Newton's laws and the laws of electrodynamics, so even in classical mechanics we have something (light) that is fundamentally wave-like.
The first major difference between classical mechanics and QM is in the vocabulary the two theories use to describe states of the world. Classical mechanics describes the states of Things by specifying their positions and velocities in 3-dimensional space (or, if we are being relativistic, in 4-dimensional spacetime). In addition to this, it assigns to each point in space a description of the forces that an object located at that location will experience. These are called fields. There are three kinds of classical fields: electric, magnetic, and gravitational. And that's it. That is all there is to classical mechanics, at least conceptually.
QM has a completely different way of describing states, a fundamentally different ontology. A quantum state is described by a mathematical object called a wave function, which is usually written as the Greek letter psi (Ψ). As you might glean from the name, the wave function is a function, which is to say, it takes an input and produces an output.
Note that I wrote a wave function rather than the wave function. You will often see "wave function" referred to with a definite article as if there is only one of them, but there isn't. There are many wave functions. Every quantum system has its own wave function. When people refer to the wave function what they usually mean is the wave function that describes the state of the entire universe. It would seem intuitively that the wave function is just the aggregate of all the individual wave functions that describe all of the particles in the universe, but this turns out not to be the case, and the reason this is not the case is one of the things that makes QM so deeply weird.
Note also that I said that the wave function is a function that takes an input and produces an output, but I didn't tell you what those inputs and outputs actually are. That's because these are not straightforward to describe. The output is not so bad: it's something called an amplitude, which is just a complex number, i.e. a number that involves the square root of negative 1, usually denoted by the letter "i". The input to the wave function is something called a point in configuration space. Configuration space is not something you are likely to have encountered in high school math, but it isn't actually very complicated. If you want to describe, say, the position of a particle in 3-dimensional physical space you can do that by writing down three numbers. If you want to describe the positions of two particles you can do that by writing down six numbers. You can then take those six numbers and treat them as if they described a point in some abstract six-dimensional space.
Why would we want make such a strange maneuver, one which requires us to imagine such a bizarre thing as a six-dimensional space? Surely that's just a mathematical curiosity which can't possibly have any physical or philosophical significance? Well, in (non-relativistic) classical mechanics that is true. As long as objects are large (relative to atoms) and slow (relative to light) we can model them with math that reflects our intuitions: objects have positions in 3-D space and velocities that are 3-D vectors, and time is an independent variable. But, as we have already seen, when things start to move fast (relative to light) that model breaks down and we can no longer treat time as an independent variable because different observers have legitimate disagreements about how fast clocks are running. So in order to get the right answers we have to treat space and time as a unified whole, as four-dimensional space-time.
But that's it. One extra dimension is already pretty mind-bending, but that's all you need for relativity. Physical objects still exist as independent entities that occupy specific locations in space-time, and so the Objective Reality Hypothesis still holds for physical objects. We can have legitimate disagreements about where and when a physical object exists, but we cannot have a legitimate disagreement that it exists. And the way that manifests itself mathematically is that, although you have to use four dimensions to describe reality, you never have to use more than four. The apparent distinction between space and time is not objectively real (it is observer-dependent) but 4-D space-time is objectively real, as are the physical objects which inhabit it.
In quantum mechanics that is no longer the case. In quantum mechanics it is impossible to ascribe independent existence to objects in 4-D spacetime and still be consistent with observation [1]. This is a famous result called Bell's theorem. It was first discovered (by John Bell) in 1964 and first demonstrated experimentally by Alain Aspect in 1982. Before that, the fact that the math of quantum mechanics had this structure was considered by many -- most notably Albert Einstein -- to be a flaw, an indication that the theory was incomplete, that something was missing. The presence or absence of an independent existence of physical objects was considered a philosophical debate, not a scientific one. That is no longer a tenable position. The fact that this mathematical structure is necessary to describe reality is now established beyond all possible doubt. The entire field of quantum computing depends on it. Since 1964 there have been even stronger results proven mathematically and demonstrated experimentally. It is inescapable: quantum mechanics is deeply weird. Specifically, using configuration space as the input to the wave function is not just a mathematical convenience. It is necessary to explain observations. Specifically, it is necessary to explain entanglement. But let's not get ahead of ourselves. Let us return instead to the elephant in the room, which is that it is obvious that what I've just told you cannot possibly be true because it is obvious that material objects do exist in point of actual fact and their existence is not dependent on observations. If a tree falls in a forest it really does make a sound even if no one is there to hear it. Chairs continue to exist even when no one is sitting in them.
Right?
So: on the one hand, QM -- and its relativistic counterpart, QFT -- are the most successful scientific theories ever devised. No experiment has ever produced results at odds with the predictions of QFT. (This is actually a major crisis in particle physics. Progress has been pretty much at a standstill for over fifty years now.) On the other hand, the mathematical structure of these theories are categorically at odds with our lived experience. How do we reconcile these two facts?
Answering that question will be our project for the next few episodes.
---
[1] Note that the way I've stated this is controversial. There is a formulation of QM that purports to ascribe independent existence to particles and even assign them definite positions and velocities. It is called the de Broglie–Bohm theory, more commonly referred to as Bohmian mechanics. But this theory has two problems. First, it is very hard to extend it to be compatible with relativity (so strictly speaking my claim that it is impossible to "ascribe independent existence to objects in 4-D spacetime and still be consistent with observation" is technically correct). But more importantly, the way in which it ascribes independent existence to particles is, in my opinion, cheating. I'm planning a whole entry about Bohmian mechanics, but I have to lay a lot more groundwork before I can discuss it cogently.