Written by Timothy Bates.
The BBC’s “Science Focus” recently published a hit piece on IQ that is wrong at almost every step.
Claire Asher, the author, argues we should “kill IQ scores for good” because IQ isn’t a perfect measure of the entirety of human capability. This is a Logic 101 failure. It’s like saying we shouldn’t take account of probabilistic information because it is “incomplete”. But in an over-determined world, saying we should ignore everything that’s not a total account of everything is an argument for complete ignorance.
Part of the problem with cognitive ability, Asher suggests, is that IQ scores simplify an incredibly complex characteristic. The truth is almost the reverse: intelligence researchers always emphasize the need to measure many different cognitive tasks. That’s why the Wechsler Adult Intelligence Scale, a major IQ test, traditionally had 15 distinct subtests. And this was recently increased to 17! So much for simplifying.
The real “simplification” was the surprising discovery that all those different cognitive tasks—from reasoning, to memory, to mental rotation and so on—correlate with each other. There is a general factor of intelligence. This g-factor grabs so much of the variance in cognition (around half) that it is nearly always the single best predictor of performance in school, at work and in life. Because it is general, it predicts things far removed from the test that was used to assess it.
Asher claims that unlike “height or weight, which are simple to measure and can be described with a single number, intelligence is a multi-faceted trait.” This exposes a misunderstanding on her part. Height too is driven by dozens of components: vertebrae length, femur length, cranial length etc. Just like the g-factor of intelligence, genes that make one body part taller tend to operate systemically. Both traits are highly heritable, highly polygenic and characterised by a general factor as well as other specific elements.
Asher notes in regard to intelligence that “scientists haven’t even been able to agree on a definition.” This old trope (which goes back to an off-hand 1923 comment by Edwin Boring that IQ is “what the tests test”) is foolish for two reasons. There is close to consensus agreement on what intelligence means. And science doesn’t work by biblical decree: researchers construct falsifiable theories about intelligence, and then test them. It would be like critiquing physics for not defining temperature at the outset—rather than puzzling over what it means, working on appropriate measures, resolving disagreements, and eventually realizing that it is the average kinetic energy of molecules in a substance.
Intelligence is a very important trait. In fact, it is the most reliably and validly measured trait in all of behavioural science. And it’s not even close. Test scores are stable over multiple decades, and the plethora of tests developed over time all converge on the same underlying trait.
To attack cognitive ability because it doesn’t measure everything about human adaptability and thriving is absurd. Again, it would be like suggesting that muscle strength is a “flawed” measure because it fails to explain everything about sporting performance. The opposite is true. By breaking performance up into distinct features linked to distinct biologies—pulmonary function, muscle strength, coordination—we can explain human performance.
The observation that IQ test scores don’t explain all human capabilities, including creativity or empathy, is not a valid criticism. It makes as much sense as saying that theories in physics unifying electricity and magnetism are flawed because they don’t explain the strong forces inside the nucleus. It’s a fundamental misunderstanding.
Asher’s claim that “while IQ tests do a good job of measuring conscious problem-solving skills, they can’t assess whether we choose to apply those skills at the right moment, rather than falling back on our instincts” is simply false. Bright people have much better intuitions, including intuitions about when a problem that appears easy will require conscious effort to solve. That’s why general ability is almost perfectly correlated with rationality: the ability to not be lured by simple but incorrect solutions.
Far from it being the case that “IQ scores are only loosely correlated with academic achievement”, intelligence correlates with academic achievement at .80 when both are measured as latent factors!
Indeed, another persistent myth from the IQ hit squad is that intelligence is just school smarts. The truth is that intelligence is the most practically relevant skill: it shines above all other measures at predicting both school performance and work performance. IQ tests are correlated with school performance among children who have never previously sat an IQ test.
The lengths of whataboutery in IQ hit pieces is, to be honest, staggering. Asher notes that IQ scores “might” become a self-fulfilling prophecy. A poor score early in life “might” influence how a child is taught, what opportunities they are given and even how they view themselves. On the other hand, a good score can allow a child to skip a class, work on more challenging material, and advance their knowledge more rapidly.
Robert Sternberg is cited as arguing that “the attributes [of intelligence] are universal. You need practical intelligence everywhere. You need creativity everywhere. But what is practical differs from one place to another.” But how much does it differ? Performance on ability tests predicts economic development across all countries.
Bright people placed in novel environments and forced to learn new ways of doing things generally excel. So much so that adaptation—coping with novelty—is core to the definition of intelligence. At its heart intelligence involves finding and storing the answers to problems. In our own society, these problems have changed dramatically, yet IQ keeps predicting who will solve them as well as it ever did. If anything, intelligence is more relevant now, due to the complexity of technology.
To suggest that abilities like strategizing, learning and remembering are not relevant outside our society is implausible. But it is testable. The phenotypic structure of g remains consistent across cultures, and as global GWAS data improves, the biological architecture will likely prove as universal as the architecture for height.
Asher mentions that “standard IQ tests” rely heavily on skills that are honed through formal education. That’s actually the inverse of what the developer of the modern IQ test, Alfred Binet, had as his goal. When the French government rolled out universal education and found that many children were struggling, they tasked him with measuring not education but educability—the ability to learn. Which is precisely why he chose materials from outside of school, materials that were present in daily life.
As for the idea that intelligence is somehow inconsistent with learning, this is fundamentally wrong. As an academic who studies intelligence, I don’t know anyone who thinks we learn nothing at school or in life. It’s a straw man. Equally, however, no one with experience (e.g., teachers) can fail to observe that children learn at dramatically different rates. These two notions—that most of what we know we learn, and that people differ in their rate of learning—are distinct and separate findings.
The hit piece then veers off from merely wrong to actively malicious. Short-circuiting the reader’s thinking for them, Asher identifies a sinister and “disquieting” side to IQ. The sin? To “compare nations’ average IQ scores”.
She concludes that scholars should not invest the “time and energy needed to collect nationally representative data on IQ” because “this data just isn’t useful”. So are the massive national differences an artefact of poor sampling, which would disappear under improved measurement? If so, it would surely be a great idea to invest the modest resources needed to check this.
Luckily, economists and the organisers of studies like PISA and TIMSS have stepped in to do just that: improve measurement of cognitive performance by collecting representative samples. Their findings align extremely closely with those of smaller IQ-based studies. Far from being “not useful”, average cognitive performance predicts national socio-economic success with a high degree of precision.
Criticisms of cross-national comparisons arise not because the tests don’t predict well—they do all too well—but instead because of what this means in terms of the causes of achievement.
Some people attribute everything to culture or environment: people are identical, but some cultures crush development. Yet the consensus when scientists are surveyed is that people differ in ability, which is both a cause of development-crushing cultures and in part caused by the culture (and in part by genes), often in a mutually reinforcing loop. This question will ultimately be resolved by biologically informed studies.
Asher then notes that “IQ has held such a prominent position for so long” because it’s easy to measure. That’s quite the acknowledgment! But of course, IQ tests have not retained the influence they have (despite concerted opposition) because they are the quickest test on the shelf (they aren’t). Rather, it’s because, for predicting future performance at school or work, they outperform everything except direct measures of prior performance on the same tasks.
People are not “suckers for things that are easy to measure”, as Sternberg would have us believe. They are choosing measures that work.
The conclusion of the hit piece represents a nadir of reasoning: “it’s unlikely that any test can ever fully capture the complexity of intelligence across the full spectrum of cultural diversity.” And “since IQ scores tell us so little and can be used to do so much harm, maybe it’s time we abandon them for good.”
Let’s take this argument seriously for a moment. If current tests struggle to capture the complexity of intelligence, the logical response would be to develop better tests that do capture this complexity.
But this is actually “business as usual” for test developers. Many test developers have tried to be culture-neutral, and the results just do not differ materially from those of other tests. They either get the same rank order results, or add bias. Personally, I would say that measures such as inspection time and reaction time have been underutilized. What’s clear is that a lot of people have expended a lot of energy exploring this potential dark matter of intelligence and have come up empty handed.
Asher’s final statement—“it’s time we abandon [IQ tests] for good”—is simply an opinion, one that’s at odds with the known facts. IQ scores out-predict all other measures of performance. Personality doesn’t come close. IQ predicts performance in the workplace. It predicts it on day one and it predicts the slope of learning across time. IQ predicts performance in manual tasks and in professional tasks. It even predicts number of patents and number of academic publications.
In short, the idea that cognitive ability “tells us so little” is about as misinformed as it is possible to be. What’s more, if we decided to abandon everything which may be “used to do harm”, IQ tests would be near the bottom of the list.
Putting aside banal examples (cars can be used to do harm), ability tests have long been one of the only objective ways for people from less advantaged backgrounds to demonstrate talent. As universities across the US are finding out, without the SAT (which functions as a very good IQ test), they can’t identify talent. Nor can they gauge whether students are capable of completing their studies.
For this purpose alone—not to mention their massive underutilized value in the workplace—we should be increasing, not abandoning, the use of IQ tests. It’s the rational thing to do.
This essay was adapted from one published here.
Timothy Bates is a professor of differential psychology at the University of Edinburgh. He has published in Intelligence, Intelligence & Cognitive Abilities and many other journals.
Become a free or paid subscriber:
Like and comment below.
Well, it's the BBC.
The BBC are fearful of the truth. It doesn’t fit their world view. Now Confas has been found crimeless, the Overton window has shifted. One can now say, ‘Blacks are two standard deviations less intelligent than whites” as a matter of fact. This position that IQ is meaningless is their new defensive trench.