Wednesday, November 07, 2007

Science 103: The virtue of simplicity

I've talked a lot so far in this series about how scientific experiments are designed and their results interpreted, about how statistics and controlled studies are used to filter out "real" results. But what does it actually mean to be a "real" result?

Here's a little puzzle to motivate the discussion: how many data points does it take to produce a statistically significant result, that is, a result that is very unlikely to have come about by chance? What is the smallest conceivable number of data points that would be needed under ideal circumstances?

Let's take a brief respite from biology and deal with physics for a moment. It is intuitively obvious that heavier objects should fall faster than lighter ones. Hold a rock and a feather in your hand and you can experience firsthand that gravity pulls harder on the rock than the feather, so it is entirely plausible that the rock should fall faster. And indeed it does (at least near the surface of the earth). This was the prevailing view among learned men (there were precious few learned women in those days) for thousands of years.

Which is interesting because even a moment's reflection will reveal that it is not intuitively obvious that heavier objects should fall faster than lighter ones. For starters, birds are heavier than feathers. Indeed, birds invariably carry a payload of tens of thousands of feathers (to say nothing of muscles and bones and other assorted support equipment) and yet if you drop a feather and a (live) bird the feather will generally fall faster. That should have been clue even to the ancients that there was something wrong with the theory that heavier objects fall faster than lighter ones. And yet, as far as I know, I am the first person ever to point this out. (One might argue that birds fall more slowly because they do work to stay aloft, but this is not the case either. Hawks can stay aloft for hours without flapping their wings.)

It gets worse. Imagine three identical rocks, two of which are coated with glue. Drop all three. Because they are identical they should fall at the same speed. Now imagine that in mid-flight the two glue-coated rocks come together and stick, making essentially a single rock that is twice as heavy. The heavier-objects-fall-faster theory would predict that this composite rock should now accelerate relative to the unglued control rock. But why should that happen if both of the component rocks were falling at the same speed to begin with? (And if that example doesn't convince you, imagine three identical skydivers. Two of them drift towards each other. Their fingers touch. They hold hands. They pull themselves towards each other and attach their harnesses together. Now they are a "composite" skydiver twice as heavy as before, and should therefore be falling faster than the lone control skydiver. At what point during this process would they start to accelerate?

As these examples illustrate, it often requires only one data point to produce a statistically significant result. Climb to the top of the leaning tower of Pisa, drop two canon balls, one twice as heavy as the other, and with a single data point you can convincingly disprove the theory that heavy objects fall faster than lighter ones.

Let's return to biology and our pink flamingos. How many non-pink flamingos would it take to disprove the theory that flamingos are genetically pink? Now it's not quite so clear. If I were to just exhibit a white flamingo one might argue that this particular bird simply has a mutation. Albino-ism is a well-known phenomenon in other species. But suppose that I showed you a white flamingo and told you that this flamingo had been raised in a zoo and fed something other than shrimp? Does that make the flamingos-are-genetically-pink theory untenable? Well, not entirely. One could still argue that this flamingo is a genetic albino, and it's just a coincidence that it was fed a non-standard diet. So then you could start feeding this flamingo shrimp and watch it turn pink. Does that make for convincing proof? Still no. A die-hard eugenicist could still argue that flamingos are genetically pink, but that the stress of being raised on food other than its natural diet somehow caused the genes for pinkness not to express themselves. Or something like that.

Of course, the heavier-objects-fall-faster theory is salvageable too if you're willing to tie yourself into enough rhetorical knots. You could argue that the heavier canon ball is also bigger and therefore experiences more drag, and that this extra drag just balances out the extra weight. Of course, this theory can also be disproved by dropping two canon balls of the same size but made of different materials. But then the die-hard Artistotelian could start spouting something about the particular materials used and how the proportion of earth to fire in their composition affects their falling rates and so on and so on. And if you think I'm belaboring the point beyond all reason, go read this or this or this or this.

There are two points to this story. First, there is no way in science to ever prove anything beyond all doubt. The best we can hope to do is to come up with parsimonious theories that are good fits to the observed data. (The fact that this is possible at all is actually quite remarkable, and is itself an observation that cries out for an explanation. Einstein once famously quipped that "the most incomprehensible thing about the Universe is that it is comprehensible." David Deutsch actually takes a pretty convincing shot at that question his book.)

Second, the number of data points that it takes to disprove a theory depends on the theory. The theory that heavier objects fall faster than lighter ones, period, end of story, can be disproved as I show above without actually conducting any experiments at all. The theory that heavier objects fall faster than lighter ones except under certain conditions is much harder to disprove, but much easier to dismiss out of hand simply because of how outlandish it seems to be a priori. Science rejects conspiracy theories not because they can be disproven (they can't -- that's why they are called conspiracy theories) but simply because they are not parsimonious. In science, simplicity is axiomatically a virtue.

In that light, Richard Lynn's theory has a lot to recommend it. It is quite parsimonious and plausible a priori. Harsh climates are indeed generally less forgiving of failures to plan ahead than milder ones. That genetics plays a significant role in determining intelligence is clear from the observation that humans are vastly more intelligent than other great apes, and the only possible explanation for that is our genes. And then there are Lynn's mountains of data, all of which seem to support the theory. It's pink flamingos as far as the eye can see.

Or is it?

In fact, there's a white flamingo in Lynn's data. Several of them actually. Some of them I've already pointed out in earlier posts so I won't belabor them here. I want to focus on one particular white flamingo: the average IQ for arctic peoples is lower than that for Europeans.

This is a serious problem for the theory that winter survival is what drives the evolution of intelligence, because if that were the case then one would expect arctic peoples to be the smartest on earth, and yet they are not by a wide margin (a full standard deviation). Lynn acknowledges this problem and dispenses with it by saying:

"The explanation for this must lie in the small numbers of the Arctic Peoples, whose population at the end of the twentieth century was only approximately 56,000 as compared with approximately 1.4 billion East Asians. While it is impossible to make precise estimates of population sizes during the main Wurm glaciation, there can be no doubt that the East Asians were many times more numerous than the Arctic Peoples. The effect of the difference in population size will have been that mutations for higher intelligence occurred and spread in the East Asians that never appeared in the Arctic Peoples.

You might want to see if you can figure out what is wrong with this argument before you proceed. I've told you everything you need to know. (Just for good measure, here's another clue.)

Lynn acknowledges a second problem:

"The Arctic Peoples did, however, evolve a larger brain size, approximately the same size as that of the East Asians, so it is curious that they do not have the same intelligence.

And dispenses with it by suggesting that the Inuit evolved "strong visual memory" that would have helped on hunting expeditions, but "which is not measured in intelligence tests."

Does this not begin to remind you of the Aristotelian trying to salvage the theory that heavier bodies fall faster?

Let us see how many problems with Lynn's little song-and-dance we can enumerate.

1. Lynn's argument that small population leads to low intelligence is circular. His entire thesis is that intelligence is an evolutionary adaption. Therefore, high intelligence leads to large populations, not the other way around. (Duh!)

2. If one admits that a small population can dominate the evolutionary pressure of a harsh environment and produce low intelligence even in the face of having to survive in winter, that same argument must then be applied to all of the data points for which the populations were small. So bye-bye to the bushmen and aborigines as supporting data points. You can't have it both ways. Either small populations produce reliable data (in which case the Arctic People's falsify the theory) or they do not, in which case Lynn's entire argument begins to come apart at the seams.

3. If small populations don't produce enough alleles for the evolutionary pressures of harsh environments to manifest themselves, where do those big brains come from, eh? You can't have it both ways. Either small populations don't manifest evolutionary pressures (in which case the Arctic People's large brains are a mystery) or they do (in which case Lynn's theory is falsified). Isn't it possible that the explanation for this discrepancy is that IQ tests don't accurately measure intelligence after all?

I'll leave it at that for now. There are in fact more holes in Lynn's theory than a Swiss cheese. But there is one gaping hole that dominates all the others: Lynn is postulating a simple theory for a complicated phenomenon, arguably the most complicated phenomenon in the entire Universe. All else being equal, simplicity is a virtue. But in this case all else is not equal. Some things are just complicated, and intelligence is one of them. Einstein once said that scientific theories should be "as simple as possible -- but no simpler." Lynn's theory is simpler, and therefore almost certainly wrong.

Intelligence is complicated. It is complicated to define. It is complicated to measure. It is produced by complicated processes that we are not even close to fully understanding. It is influenced by many disparate factors. Genes are undoubtedly among those factors, and it is a valid question to inquire into the extent to which genes contribute to overall intelligence (whatever that means). But -- and this is the crucial point -- Lynn does not answer that question! The reason he doesn't answer it is that he doesn't ask it. He assumes that the answer is "a lot" and goes on to ask a different question, namely, how much correlation is there between the genes that make us intelligent and the genes that make us members of our respective ethnic groups. Then, having asked the wrong question, he then goes on to make just about every mistake in the book, including collecting a mountain of data and drawing conclusions from analysis that is both post hoc and ad hoc.

I don't know what prompted James Watson to make the remarks that he did about black people, but by no stretch of the imagination are his remarks defensible as reasonable interpretations of currently available scientific data. At best, the jury is still out.

There is one final item I want to address. I can't find it at the moment, but someone left a comment on one of these posts to the effect that I "want" Lynn's theory to be wrong, that I want it to turn out that there are no racial differences in intelligence. That is true. I do hope it turns out that Lynn is wrong because I have seen the great evils that result when people believe that Lynn is right even in the absence of evidence. I think it would be a great tragedy if science were to give solace to bigots and white supremacists, and it is possible that that desire has colored or biased my evaluation of Lynn's work. I've done my best to be objective, but I am only human.

I will say (or maybe I should say "confess") that I did feel a certain sense of relief when I read Lynn's book and found it fatally flawed. There are certain inquiries for which it is wise, before they are undertaken, to think about what one is going to do with the knowledge once it is acquired, and to consider the possibility that there may be things that we would be better off not knowing.

denis bider said...

"This is a serious problem for the theory that winter survival is what drives the evolution of intelligence, because if that were the case then one would expect arctic peoples to be the smartest on earth, and yet they are not by a wide margin (a full standard deviation). Lynn acknowledges this problem and dispenses with it"

I too find his speculation in this instance somewhat inept and disingenious. Obviously, the arctic people have adapted to their environment, even developed "strong visual memory", but not developed a higher g.

But the value of Lynn's book is not his speculations - it is the collection of data. If his speculations are bad, I don't think that affects the value or significance of the data.

For the Arctic people in particular, I would have thought of a much less self-defeating explanation than you quote. The Arctic people face the same environment year-round, much like the people in sub-Saharan Africa. When the environment remains the same in the long run, this means that people can adapt to it in more straightforward, more environment-specific ways than by increasing general intelligence. Thus, sub-Saharan Africans developed longer limbs and better developed muscles, and the Arctic people developed strong visual memory and other things.

Meanwhile, the Europeans faced a different environment during the summers than they did during winters, and they had to survive during both. This means that environment-specific adaptations were less useful because the environment varied over the course of the year, which means that increases in general intelligence were more useful.

That seems like a perfectly valid theory; simpler, nicer, it is more general, and it fits more data points.

Of course, like you already stressed before, just because I have a theory that explains the observations, that doesn't mean that it is true. But the reason I state it is to show that just because you are able to find shortcomings in Lynn's speculations, doesn't mean that you're disproving anything. Just because Lynn draws genetic interpretations that are inconsistent, it doesn't mean that genetic interpretations that fit the data and are consistent don't exist.

This is as opposed to environmental interpretations of the same data, in which case I can't think of one that would be compatible with the same data set.

"There are certain inquiries for which it is wise, before they are undertaken, to think about what one is going to do with the knowledge once it is acquired, and to consider the possibility that there may be things that we would be better off not knowing."

Nah. I am very disinclined towards that attitude. We seek knowledge because it's interesting, period. Besides the pursuit of knowledge and power, all that's left for us to do is to eat, sleep, exercise and procreate. The choice is either more knowledge or stagnation. Given that choice, I'm always in favor of more knowledge.

denis bider said...

"Besides the pursuit of knowledge and power"

This could be misread. "Power" here does not necessarily mean power over other people, but the ability to manipulate reality, the kind of power that derives more or less directly from knowledge; the ability to get more bang for the same buck, more effect for the same input; initially to have to work less and have more time to eat, sleep, exercise and procreate, and subsequently, once this has been obtained, as a creative activity in and of itself, possibly with goals such as to cure disease, explore the galaxy, prolong lifespan.

Ron said...

The choice is either more knowledge or stagnation.

I agree. But our resources are finite so we have to prioritize the inquiries we make anyway. I'm just saying that taking the potential consequences of having answers ought to be taken into account when we make those choices.

denis bider said...

But our resources are finite so we have to prioritize the inquiries we make anyway. I'm just saying that taking the potential consequences of having answers ought to be taken into account when we make those choices.

In the absence of totalitarianism, the way resources are prioritized is dictated by decisions of individuals. I believe it is incorrect to say that "the potential consequences of having answers" are not already accounted for in the decisions taken by these individuals. There are significant potential consequences in understanding intelligence. The questions of why African IQs are depressed or how the Flynn effect takes place seem like pretty important pieces of the puzzle. I don't believe that you can easily solve a puzzle of this magnitude by ignoring these enormous questions.

Basically you are suggesting that the benefit obtained by "not encouraging" white supremacists exceeds the benefit obtained by understanding intelligence. I find that unlikely and "dark-ageist", in a sense.

Furthermore, even before we arrive at a comprehensive understanding of intelligence in general, there are significant benefits from understanding the cause of African IQ depression on its own. The policy decisions we make are an obvious example. Knowing that the mechanism is predominantly environmental or that the mechanism is predominantly genetic, we can focus on those policies which are compatible with actual reality, rather than those other policies which would have worked if only reality were different.

Such understanding can save society trillions and provide a better world for everyone.

denis bider said...

Damn, man - to think about it, you really sound like a medieval Catholic preacher. "Thou shalt not read from the Holy book unless you read it in Latin! And if you cannot read it in Latin, thou shalt ask the preacher to interpret it for you!" The holy book is human knowledge, and preacher are "enlightened" people who are qualified to "interpret" it, like you. Among the holloi polloi are the "white supremacists" and "bigots" who should be excluded from knowledge so that they don't misinterpret it, and these bigots are apparently so frequent among the masses that knowledge should be denied not only to them, but to everyone.

I find it amusing that you can have these beliefs and yet simultaneously call yourself liberal. :-)

Ron said...

I find it amusing that you can have these beliefs and yet simultaneously call yourself liberal.

I find it amusing that you don't seem to know the meaning of the word "simultaneously." It has been quite a while since I last called myself a liberal.

And not that I really want to dignify this latest ad hominem attack with a response, but just because I think that a certain course of action is unwise does not mean that do not recognize people's right to pursue that course of action notwithstanding my reservations. People have a right to do all manner of foolish things. That doesn't make them any less foolish.

denis bider said...

Sorry, I got carried away. Again, not meant to be an attack, but rather an observation. It seemed compelling when I wrote it. :)

In retrospect it does appear that I got carried away and concluded in a manner that, at best, seems to have a tenuous relationship with what you had written.

asdf said...

to enrich this discussion, check out feynman's discussion (starting at 3:18) on how difficult it is to *know* something, how easy is to make mistakes and fool yourself.