I think it was Charles Murray who compared IQ scores to the weight of offensive lineman - there's a minimum threshold that you need to meet to really compete at the higher levels, but once you pass that threshold, more does not correlate at all with higher performance. Winning at those higher levels requires those things that are difficult or impossible to measure, the intangibles, that being "on fire with thought"... or to carry on the football analogy, being on fire with pushing those other dudes out of the way.
This is false. The flaw in that kind of analysis is restriction of range. If you sample from the cream of the crop, the correlation will weaken substantially, but If you sample from a broader population (not just the top .1% or whatever) you’ll see that there’s still a strong correlation between measures of ability on the one hand and performance on the other. (For another example, look at the work of David Lubinski, who studies precocious youth: https://www.vanderbilt.edu/psychological_sciences/bio/david-lubinski)
“If you sample from the cream of the crop, the correlation will weaken substantially”
That’s precisely the point. I’m specifically making a claim here only about the cream of the crop. The measures of performance are effective for determining who qualifies as “cream of the crop.” But within the cream of the crop subgroup, scoring higher on the performance measure does not correlate strongly or perhaps at all with higher performance outcomes.
The range restriction error to which you refer would be to conclude that because there is weak correlation between the performance measure and performance outcomes among the cream of the crop, the performance measure is invalid and useless. But that’s not at all what I’m saying here.
Sorry, I dashed that comment off too quickly. Allow me to revise it. See the following paper: https://journals.sagepub.com/doi/10.1177/0016986220925447. Ctrl+f “question 1”, and you’ll see the following: “Is there an ability threshold, beyond which more ability doesn’t matter? No.
While other things are certainly required for all important life accomplishments, greater ability leads to greater achievement.” And “The top quartile of the top 1% is at much more promise for these accomplishments than the bottom quartile of the top 1%, even though the latter are gifted and performed well beyond normative base rate expectations for these rare outcomes.”
What I had in mind when I wrote my original comment was a type of study that looks at, say, the undergrads at Caltech and compares their SAT scores to measures of academic success. (Shockingly, Caltech itself conducted such a study last year and used it as grounds for ditching the SAT/ACT.) Since the students pretty much all have perfect scores, SAT is a weak predictor of success for that population. The problem is that the SAT can’t distinguish well between the top 1%, the top .1%, and the top .01% when it comes to 17/18 year old students. However, giving it to 13 year olds is a great way of identifying the extremely gifted, as it’s much harder to get a top score at that age than it is a few years later.
As for Murray, I’m pretty sure he’s written against the idea that the utility of intelligence maxes out at a certain level. I vaguely recall first encountering the argument in either TBC or one of his essays.
Mr. George, The evidence you cited does not support the claim, because the SAT, by design, is game-able. The SAT prep industry makes billions. At the top 1% level, the relative predictive power of IQ drastically diminishes compared to how many hours of deliberate prep they did. It's not surprising that more studious kids become PhDs. It's also not surprising that the author didn't even attempt to control for the fact that IQ's correlation with SAT within the top 1% (of iq) is laughably nonexistent.
Did you look at the study? They were not looking at high school-age test takers, but rather 13 year olds. Taking the SAT at age 13 is drastically different from taking it at age 17. The correlation with IQ is definitely not laughably nonexistent. I’d encourage you to look further at Lubinski’s work, as he’s one of the top experts in this particular area.
The SAT is not really “gameable”. I tutored the SAT and ACT for several years and worked with a pretty broad range of students. There are basically two ways students can increase their scores. The first is getting more familiar with the test: the format, the time limits, and the types of questions that tend to come up. The second is learning mathematical and grammatical material that they don’t already know. It doesn’t make sense to call either of these things “gaming the test” - it’s more sensible to consider a student’s first score an underestimate of their ability due to lack of familiarity with the test format/lack of certain prior knowledge (e.g. punctuation rules; certain theorems that are easily forgotten if you don’t use them). The correlation with IQ is about 0.75-0.8, which shows that there isn’t that much room for gaming.
The test prep industry is kind of a scam, to be honest. Hiring a tutor is similar to hiring a personal trainer. It’s a luxury service that’s useful for keeping you accountable when training and for teaching you certain things - but none of those things are secret knowledge that only tutors have. A diligent student could easily get the same information for free online. To reiterate, these things are not “tricks”, but rather grammatical rules, mathematical content, and pretty obvious tips like “make sure you understand what the question is asking.” You don’t have to take my word for it though - just look up studies that estimate the effect of private tutoring. It’s something like +10 points on the SAT, which is basically zero.
I appreciate your touching on the middle ground between "IQ tests are gospel" and "IQ is made up." It seems pretty clear to me that: a) people differ in their intellectual abilities in ways that are at least somewhat innate (though it's also possible to excel or lag in some areas but not others); b) it's not entirely innate; and c) it's hard to have an entirely unbiased test intellectual ability, let alone tease out innate from acquired ability, particularly in a given person. However, there seem to be a lot of people who like to use (b) and (c) to "prove" (a) isn't true.
I agree with the main point of this post (attempts to judge famous people's IQs are garbage, IQ of 160 is basically unmeasurable), but I think along the way it gets some things importantly wrong.
1. It uses the word IQ in a way that equivocates between "score on an IQ test" and "g, the thing IQ tests measure". Many of its points don't make sense once you remove that equivocation. For example, it's not true that nobody's IQ is 160. Certainly some people get scores of 160 on IQ tests. And some people are four standard deviations from the mean on g. It's just that they're probably not the same people, or we can't say with confidence that they are.
2. Likewise, when you say "IQ is changeable", you mean "score on IQ tests". This is no more interesting than saying that height "is changeable" because you can stand on your tiptoes when they're measuring you, or wear platform shoes. g doesn't seem to be changeable, or at least it would take much more evidence to convince me that it was.
3. The part about benefits of high IQ vanishing past a certain point is probably wrong. The study you cite has some issues and is contradicted by other studies; see Link 23 at https://astralcodexten.substack.com/p/links-for-march-2023 for directions to some other places that can explain the problems better than I can.
4. I don't have an airtight retort to the Einstein story except to ask whether you think it would be fair to say Einstein "wasn't very good at math". "Good at math" seems to be the sort of thing which is heavily correlated with math grades and math test scores, in the same way that mathematical IQ is correlated with math grades and math test scores. So I see only two options. First, Einstein wasn't very good at math (this doesn't seem impossible to me, maybe he just worked really hard, got enough math to do physics at all, and was truly exceptional in some other area like creativity). Second, he was good at math, but he found his classes boring or oppressive, didn't try very hard, and so his grades don't reflect his talent very well. I think switching from the mystical-sounding word "IQ" to the normal-sounding words "good at math" help prime our intuition here and make the second possibility seem pretty likely. See https://www.openculture.com/2020/04/albert-einsteins-grades-a-fascinating-look-at-his-report-cards.html for more evidence that this is true.
That having been said, I think Kasparov had a real IQ test and it was 135. It wouldn't surprise me if top-in-the-world-at-something geniuses often had about IQ 135 because their skill is only modestly correlated with IQ. My guess is that some geniuses will be around 135 (because they're using something like creativity to succeed) and others will be higher (because they're using raw intelligence).
Hopefully someone smarter than me will come along soon to address some of the other points.
Much appreciate you taking the time, Scott. I think you’re right - in terms of general thrust we agree (most geniuses don't have bank-breaking IQs), but let’s see if we can differentiate at all between our positions, so close as they are are on the spectrum, or if it all turns out to be merely emphasis.
(1) If we instead talk about g rather than IQ, we see a similar debate. Does it matter for the center of the distribution? Yup. Does it provide differentiable information at very high scores? The original proposer of g literally had a whole thing about how higher scores on different cognitive tests were correlated less and so g diminished in importance (or at least, measurability). Found this more recent paper talking about it: https://psycnet.apa.org/record/2017-40946-001.
(2) Same as above: if we assume g is a real construct as definite as height, sure, but if we think of it as suffering a lot of the same problems in measurability and lack of correlation to the real world at high levels, etc, then substituting it in doesn’t help for clarity. In fact, I’m pretty sure g is classically supposed to be over a huge range of things, including like, musical ability, which seems to obviously stop correlating to intelligence at a certain point.
(3) I actually don’t think that particular study about income vs. IQ is “probably wrong.” In the link you’re referring to, I didn’t find any of the blogs arguing against it convincing. They often open with something like “but the Terman study of genius” and yet, I almost used that very study of an example of IQ not mattering! A bunch of the subjects ended up with normal jobs, they weren’t much different in outcomes from others from their family backgrounds, etc. Terman even did things like write his own subjects letters of recommendation, interfere in their lives, etc. Instead, I examined a modern contradictory study from the actual literature itself (the “Can you ever be too smart for your own good?” study) and my thought was that it wasn’t nearly as strong as the authors made it out to be. More broadly, just speaking at a meta-level, it’s sort of surprising to me that intelligence research - a highly political field, an obviously sensitive issue - is spared (not by all, but by a diehard cohort) the general skepticism around replicability and effect sizes that plague the rest of psychology. If ego depletion, the Stanford prison experiment, and the Dunning-Kruger effect, all things you can still find in modern psych 101 textbooks, are all wrong (as is likely so), I think it's therefore unlikely IQ research isn't touched by this, especially these older studies on genius (the Roe book is a good example of this - still trotted out occasionally, but deeply flawed).
(4) Well, I certainly think it's fair to say Einstein was good at math by the time he was making his discoveries! By definition Einstein’s incredible future of reshaping physics wasn’t reflected in his very early record, which is merely that of a smart young man. . . but I don't see any reason to believe his talent would have been captured by an early IQ test either (like he wouldn’t score 160 on it if you could somehow teleport back and give young Einstein one). I understand your point about swapping in "good at math" for "IQ" but I think Einstein got obviously better and better at math and physics throughout his life. What does that do to the intuition pump here? Would we then say that his IQ was increasing when we swap it back to make the argument? Or that we're speculating beyond the bounds of falsifiability about some entity we have no access to, and asking it to explain Einstein's success? All merely hypothetical questions - my point is that of course I think Einstein was a genius and good at math, it's just that I don't think the spark that didn't correlate to his grades early on was his IQ.
Interesting to hear about Kasparov, I didn't know that. Makes sense - I totally agree with you that most world-historic geniuses would score below the “genius line" of IQ.
Scott, I have very good math but just about anyone can beat me at chess. I have a rubbish spatial IQ. Also despite being put up a year, school was too easy and I did just enough to get by. (If really motivated I would start studying the afternoon before an exam instead of the night before.) Tiger Moms could actually be a very good thing!
Well, this was triggering. I test well, but perform badly. This has been a curse. Nobody should ever give a child an IQ test which will follow her like a stench throughout her schooling, provoking standoffs between parents and teachers blaming one another for her failure to reach her full potential. And don’t get me started on the smug Gould; for awhile, he was everywhere I went, and you’d have to either leave immediately or be stuck like a hostage, listening to him for hours. I’m still not sure if people really thought he was a genius, or if his agent was the actual genius.
The issue of motivation is salient to me. I'm not surprised that Einstein did poorly or unexceptionally at school. Maybe he just wasn't interested. I did poorly in most of my teen years -- lots of D and C-s, with an occasional B and one high score when biology moved from insects (boring!) to humans (interesting!). I had to re-do my last year (O-level time in England). However, once I became fascinated with philosophy, politics, and economics, I suddenly went to the top of my classes, was allowed to take the Oxbridge exams, got it, and ended up with a doctorate. Interest and motivation was *extremely* important. Some people seem different and can make themselves study things they find boring. Not me. Maybe not Einstein.
I honestly think that, above that minimum threshold, ability to maintain motivation on boring topics has got to be one of the key differentiators for high earners (here I mean 'normal people who become millionaires mainly through accumulated wages', billionaires happen for different reasons than millionaires do).
I'm moderately intelligent, but I'm certainly not 'on fire with thought'.
Do I *love* delving deep into excel models to calculate debt ratios and then close-read hundreds of pages of legal documents to make sure the mechanics in the model match the mechanics in the legal documentation? Nope! Is it very remunerative? You bet it is, as long as that ability to focus is matched with adequate social skills.
To add some anecdata to the practice point, when applying for finance jobs, I had a lot of time (being unemployed) and a lot of motivation (being poor and was going to be kicked out of the country if I didn't get a job ASAP) so I practiced the tests for about eight hours a day for a few weeks. These maths/logic/language tests are thinly veiled IQ tests, and my scores increased dramatically over those few weeks.
I thought this was an excellent essay, and very well outlines the problems with IQ testing. I had to take a class in assessment for my graduate degree, and understanding the pros and cons of each is helpful. Unfortunately, people utilize these tests as a marker of ability or skill, and it seems fair to say that IQ is more of a test of potential, not actualized skill. Unpopular opinion time, but some of the smartest people I've meet with degrees don't have a lick of common sense, even to problem solve the simplest of tasks; people with life experience after hard setbacks seem to demonstrate better ability to navigate the world than people who rely solely on their intelligence. Often when I assess clients during BPS assessments (biopsychosocial) and through subsequent client work, I work to determine their level of insight, and understand their capacity vs their ability. As you noted, practicing on problem solving for types of problems teaches the skills of solving the problem, hence leading to greater ability to take the test, and not a true demonstration of actual skill.
Malcom Gladwell's "Outliers" I think presents a better case for achievement through having peers and mentors who teach the navigation of the social world as a greater marker for upward advancement than strictly IQ. Langan is mentioned as a case study in the book that just because you have a high IQ doesn't necessarily mean you will advance in life.
While we're obsessing over those on the right-hand side of the X axis, we should spare a thought for those on the left, close to the origin, those in the death-penalty states for whom the only bragging rights from a single point gained may be the right to choose your last meal.
If Eric's critique is on the money, and I think it is, then investing IQ tests with the power and precision to determine life or death, even if in only a small number of cases, is obscene.
In the continuing absence of any worthwhile definition of intelligence, I shall continue to regard IQ tests as measuring the ability of people to do IQ tests.
That's a really good point - I haven't looking to if the variance also increases the other way, but I'd assume it does. And even if it doesn't, and it's still, say, 10 point spreads, that could make someone qualify or not for the death penalty depending on what day they take the test, what test they take, etc.
But since you had enough time to read the essay and provide a haughty-sounding dismissal: Perhaps you can provide one or two examples of those numerous errors?
I really liked this essay, especially the line "they were people on fire with thought."
It's also really cool how I start to see threads between different pieces of yours (and other authors) here. Clearly this piece speaks to your piece on the SAT, which you mention, but also to both your essay Why we stopped making Einstein's and, in my mind, to an essay that Adam Mastroianni published recently making the argument that science is fundamentally a strong-link problem.
My background is in the humanities and I work on the line between social science and public policy today, so I am clearly biased in my own way, but I feel that the obsession with IQ speaks to a larger desire to quantify the world, which, in its myopic insistence that everything can be understood with numbers, loses the thread that the qualitative aspects of our world also matter.
Occasionally you have a really strong prior that influences the way you interface with the world on a practical level. This completely dismantled a very strong prior I had about the way IQ scores determine outcomes. My IQ was “tested” in the 120s as a kid, and I scored 2000 on my SAT w/ no prep courses or anything. I thought of myself as smart, but would never even put myself in the same universe as someone like Richard Feynman. I kind of thought this was a done deal. I think to some extent I’ve sold myself short. Obviously the point here is that IQ is just one of many components that goes in to “brain power”, and I have nowhere near the “brain power” of someone like Richard Feynman. But I think reading this will actually encourage me to test my limits a little bit harder.
Thanks Eric. Highlighted some problems of IQ testing. I am to present to my lab about the controversies of IQ testing (race x IQ = nightmare), and you have provided me some criticisms against the process itself. It also made me think of my clinical supervisor, and how she speaks about her daughter. She will make off-handed comments about her daughters IQ (145!) without adding that all her students (12 of us) use her daughter as practice for administering the WISC-V - a standardised intelligence test for children. I never have the heart to repeat back to her what she always tries to remind us about psychometrics: "Practice effects, like, they influence stuff, scores and all that. Just be careful."
There's a lot of talk about IQ these days in Substack and Podcasts but not enough about Nurture / Culture. In their Triple Pack book Amy Chua and Jed Rubenfeld talk about how "Chinese Americans are getting more bang for their intelligence buck. Chinese Americans with an IQ of (say) 103 get significantly better grades in school, scores on tests, and ultimately higher-paying jobs than do white Americans with an IQ of 103 "
Interesting read. Strange though, the one thing I took from it is that while IQ gives us some indicator of intelligence (which is a highly contested notion as it is) I wonder is it any better than your realization of how intelligent your colleagues are just by your own interaction with them. You didn’t need any IQ to tell you that. Is that experiential realization a better indicator than an IQ test?
i think it'd be really interesting to try to look at metrics surrounding problem-solving capabilities when answers themselves are not well-defined, as i believe the ability to tackle the unknown and to see through the fog is essential to what many consider to be "genius". with something like an IQ test or an SAT test you lose some of this critical info because as you note you can get better at test-taking by simply practicing test-taking. so it's like intelligence is correlated with IQ but there is a large dispersion on that correlation. we have to figure out what key factors drive that dispersion/if there is an alternative metric that correlates with less dispersion.
Stephen Hsu has a fairly nuanced ongoing discussion of the Study of Mathematically Precocious Youth, which tracks the life histories of a sample of very high-IQ youths. Granting all the caveats, it's still striking how much can be predicted by the results of taking an IQ test at a young age: https://infoproc.blogspot.com/search/label/smpy
I think it was Charles Murray who compared IQ scores to the weight of offensive lineman - there's a minimum threshold that you need to meet to really compete at the higher levels, but once you pass that threshold, more does not correlate at all with higher performance. Winning at those higher levels requires those things that are difficult or impossible to measure, the intangibles, that being "on fire with thought"... or to carry on the football analogy, being on fire with pushing those other dudes out of the way.
This is false. The flaw in that kind of analysis is restriction of range. If you sample from the cream of the crop, the correlation will weaken substantially, but If you sample from a broader population (not just the top .1% or whatever) you’ll see that there’s still a strong correlation between measures of ability on the one hand and performance on the other. (For another example, look at the work of David Lubinski, who studies precocious youth: https://www.vanderbilt.edu/psychological_sciences/bio/david-lubinski)
“If you sample from the cream of the crop, the correlation will weaken substantially”
That’s precisely the point. I’m specifically making a claim here only about the cream of the crop. The measures of performance are effective for determining who qualifies as “cream of the crop.” But within the cream of the crop subgroup, scoring higher on the performance measure does not correlate strongly or perhaps at all with higher performance outcomes.
The range restriction error to which you refer would be to conclude that because there is weak correlation between the performance measure and performance outcomes among the cream of the crop, the performance measure is invalid and useless. But that’s not at all what I’m saying here.
Sorry, I dashed that comment off too quickly. Allow me to revise it. See the following paper: https://journals.sagepub.com/doi/10.1177/0016986220925447. Ctrl+f “question 1”, and you’ll see the following: “Is there an ability threshold, beyond which more ability doesn’t matter? No.
While other things are certainly required for all important life accomplishments, greater ability leads to greater achievement.” And “The top quartile of the top 1% is at much more promise for these accomplishments than the bottom quartile of the top 1%, even though the latter are gifted and performed well beyond normative base rate expectations for these rare outcomes.”
What I had in mind when I wrote my original comment was a type of study that looks at, say, the undergrads at Caltech and compares their SAT scores to measures of academic success. (Shockingly, Caltech itself conducted such a study last year and used it as grounds for ditching the SAT/ACT.) Since the students pretty much all have perfect scores, SAT is a weak predictor of success for that population. The problem is that the SAT can’t distinguish well between the top 1%, the top .1%, and the top .01% when it comes to 17/18 year old students. However, giving it to 13 year olds is a great way of identifying the extremely gifted, as it’s much harder to get a top score at that age than it is a few years later.
As for Murray, I’m pretty sure he’s written against the idea that the utility of intelligence maxes out at a certain level. I vaguely recall first encountering the argument in either TBC or one of his essays.
Mr. George, The evidence you cited does not support the claim, because the SAT, by design, is game-able. The SAT prep industry makes billions. At the top 1% level, the relative predictive power of IQ drastically diminishes compared to how many hours of deliberate prep they did. It's not surprising that more studious kids become PhDs. It's also not surprising that the author didn't even attempt to control for the fact that IQ's correlation with SAT within the top 1% (of iq) is laughably nonexistent.
Did you look at the study? They were not looking at high school-age test takers, but rather 13 year olds. Taking the SAT at age 13 is drastically different from taking it at age 17. The correlation with IQ is definitely not laughably nonexistent. I’d encourage you to look further at Lubinski’s work, as he’s one of the top experts in this particular area.
The SAT is not really “gameable”. I tutored the SAT and ACT for several years and worked with a pretty broad range of students. There are basically two ways students can increase their scores. The first is getting more familiar with the test: the format, the time limits, and the types of questions that tend to come up. The second is learning mathematical and grammatical material that they don’t already know. It doesn’t make sense to call either of these things “gaming the test” - it’s more sensible to consider a student’s first score an underestimate of their ability due to lack of familiarity with the test format/lack of certain prior knowledge (e.g. punctuation rules; certain theorems that are easily forgotten if you don’t use them). The correlation with IQ is about 0.75-0.8, which shows that there isn’t that much room for gaming.
The test prep industry is kind of a scam, to be honest. Hiring a tutor is similar to hiring a personal trainer. It’s a luxury service that’s useful for keeping you accountable when training and for teaching you certain things - but none of those things are secret knowledge that only tutors have. A diligent student could easily get the same information for free online. To reiterate, these things are not “tricks”, but rather grammatical rules, mathematical content, and pretty obvious tips like “make sure you understand what the question is asking.” You don’t have to take my word for it though - just look up studies that estimate the effect of private tutoring. It’s something like +10 points on the SAT, which is basically zero.
I appreciate your touching on the middle ground between "IQ tests are gospel" and "IQ is made up." It seems pretty clear to me that: a) people differ in their intellectual abilities in ways that are at least somewhat innate (though it's also possible to excel or lag in some areas but not others); b) it's not entirely innate; and c) it's hard to have an entirely unbiased test intellectual ability, let alone tease out innate from acquired ability, particularly in a given person. However, there seem to be a lot of people who like to use (b) and (c) to "prove" (a) isn't true.
I agree with the main point of this post (attempts to judge famous people's IQs are garbage, IQ of 160 is basically unmeasurable), but I think along the way it gets some things importantly wrong.
1. It uses the word IQ in a way that equivocates between "score on an IQ test" and "g, the thing IQ tests measure". Many of its points don't make sense once you remove that equivocation. For example, it's not true that nobody's IQ is 160. Certainly some people get scores of 160 on IQ tests. And some people are four standard deviations from the mean on g. It's just that they're probably not the same people, or we can't say with confidence that they are.
2. Likewise, when you say "IQ is changeable", you mean "score on IQ tests". This is no more interesting than saying that height "is changeable" because you can stand on your tiptoes when they're measuring you, or wear platform shoes. g doesn't seem to be changeable, or at least it would take much more evidence to convince me that it was.
3. The part about benefits of high IQ vanishing past a certain point is probably wrong. The study you cite has some issues and is contradicted by other studies; see Link 23 at https://astralcodexten.substack.com/p/links-for-march-2023 for directions to some other places that can explain the problems better than I can.
4. I don't have an airtight retort to the Einstein story except to ask whether you think it would be fair to say Einstein "wasn't very good at math". "Good at math" seems to be the sort of thing which is heavily correlated with math grades and math test scores, in the same way that mathematical IQ is correlated with math grades and math test scores. So I see only two options. First, Einstein wasn't very good at math (this doesn't seem impossible to me, maybe he just worked really hard, got enough math to do physics at all, and was truly exceptional in some other area like creativity). Second, he was good at math, but he found his classes boring or oppressive, didn't try very hard, and so his grades don't reflect his talent very well. I think switching from the mystical-sounding word "IQ" to the normal-sounding words "good at math" help prime our intuition here and make the second possibility seem pretty likely. See https://www.openculture.com/2020/04/albert-einsteins-grades-a-fascinating-look-at-his-report-cards.html for more evidence that this is true.
That having been said, I think Kasparov had a real IQ test and it was 135. It wouldn't surprise me if top-in-the-world-at-something geniuses often had about IQ 135 because their skill is only modestly correlated with IQ. My guess is that some geniuses will be around 135 (because they're using something like creativity to succeed) and others will be higher (because they're using raw intelligence).
Hopefully someone smarter than me will come along soon to address some of the other points.
Much appreciate you taking the time, Scott. I think you’re right - in terms of general thrust we agree (most geniuses don't have bank-breaking IQs), but let’s see if we can differentiate at all between our positions, so close as they are are on the spectrum, or if it all turns out to be merely emphasis.
(1) If we instead talk about g rather than IQ, we see a similar debate. Does it matter for the center of the distribution? Yup. Does it provide differentiable information at very high scores? The original proposer of g literally had a whole thing about how higher scores on different cognitive tests were correlated less and so g diminished in importance (or at least, measurability). Found this more recent paper talking about it: https://psycnet.apa.org/record/2017-40946-001.
(2) Same as above: if we assume g is a real construct as definite as height, sure, but if we think of it as suffering a lot of the same problems in measurability and lack of correlation to the real world at high levels, etc, then substituting it in doesn’t help for clarity. In fact, I’m pretty sure g is classically supposed to be over a huge range of things, including like, musical ability, which seems to obviously stop correlating to intelligence at a certain point.
(3) I actually don’t think that particular study about income vs. IQ is “probably wrong.” In the link you’re referring to, I didn’t find any of the blogs arguing against it convincing. They often open with something like “but the Terman study of genius” and yet, I almost used that very study of an example of IQ not mattering! A bunch of the subjects ended up with normal jobs, they weren’t much different in outcomes from others from their family backgrounds, etc. Terman even did things like write his own subjects letters of recommendation, interfere in their lives, etc. Instead, I examined a modern contradictory study from the actual literature itself (the “Can you ever be too smart for your own good?” study) and my thought was that it wasn’t nearly as strong as the authors made it out to be. More broadly, just speaking at a meta-level, it’s sort of surprising to me that intelligence research - a highly political field, an obviously sensitive issue - is spared (not by all, but by a diehard cohort) the general skepticism around replicability and effect sizes that plague the rest of psychology. If ego depletion, the Stanford prison experiment, and the Dunning-Kruger effect, all things you can still find in modern psych 101 textbooks, are all wrong (as is likely so), I think it's therefore unlikely IQ research isn't touched by this, especially these older studies on genius (the Roe book is a good example of this - still trotted out occasionally, but deeply flawed).
(4) Well, I certainly think it's fair to say Einstein was good at math by the time he was making his discoveries! By definition Einstein’s incredible future of reshaping physics wasn’t reflected in his very early record, which is merely that of a smart young man. . . but I don't see any reason to believe his talent would have been captured by an early IQ test either (like he wouldn’t score 160 on it if you could somehow teleport back and give young Einstein one). I understand your point about swapping in "good at math" for "IQ" but I think Einstein got obviously better and better at math and physics throughout his life. What does that do to the intuition pump here? Would we then say that his IQ was increasing when we swap it back to make the argument? Or that we're speculating beyond the bounds of falsifiability about some entity we have no access to, and asking it to explain Einstein's success? All merely hypothetical questions - my point is that of course I think Einstein was a genius and good at math, it's just that I don't think the spark that didn't correlate to his grades early on was his IQ.
Interesting to hear about Kasparov, I didn't know that. Makes sense - I totally agree with you that most world-historic geniuses would score below the “genius line" of IQ.
Scott, I have very good math but just about anyone can beat me at chess. I have a rubbish spatial IQ. Also despite being put up a year, school was too easy and I did just enough to get by. (If really motivated I would start studying the afternoon before an exam instead of the night before.) Tiger Moms could actually be a very good thing!
Well, this was triggering. I test well, but perform badly. This has been a curse. Nobody should ever give a child an IQ test which will follow her like a stench throughout her schooling, provoking standoffs between parents and teachers blaming one another for her failure to reach her full potential. And don’t get me started on the smug Gould; for awhile, he was everywhere I went, and you’d have to either leave immediately or be stuck like a hostage, listening to him for hours. I’m still not sure if people really thought he was a genius, or if his agent was the actual genius.
The issue of motivation is salient to me. I'm not surprised that Einstein did poorly or unexceptionally at school. Maybe he just wasn't interested. I did poorly in most of my teen years -- lots of D and C-s, with an occasional B and one high score when biology moved from insects (boring!) to humans (interesting!). I had to re-do my last year (O-level time in England). However, once I became fascinated with philosophy, politics, and economics, I suddenly went to the top of my classes, was allowed to take the Oxbridge exams, got it, and ended up with a doctorate. Interest and motivation was *extremely* important. Some people seem different and can make themselves study things they find boring. Not me. Maybe not Einstein.
I honestly think that, above that minimum threshold, ability to maintain motivation on boring topics has got to be one of the key differentiators for high earners (here I mean 'normal people who become millionaires mainly through accumulated wages', billionaires happen for different reasons than millionaires do).
I'm moderately intelligent, but I'm certainly not 'on fire with thought'.
Do I *love* delving deep into excel models to calculate debt ratios and then close-read hundreds of pages of legal documents to make sure the mechanics in the model match the mechanics in the legal documentation? Nope! Is it very remunerative? You bet it is, as long as that ability to focus is matched with adequate social skills.
To add some anecdata to the practice point, when applying for finance jobs, I had a lot of time (being unemployed) and a lot of motivation (being poor and was going to be kicked out of the country if I didn't get a job ASAP) so I practiced the tests for about eight hours a day for a few weeks. These maths/logic/language tests are thinly veiled IQ tests, and my scores increased dramatically over those few weeks.
I thought this was an excellent essay, and very well outlines the problems with IQ testing. I had to take a class in assessment for my graduate degree, and understanding the pros and cons of each is helpful. Unfortunately, people utilize these tests as a marker of ability or skill, and it seems fair to say that IQ is more of a test of potential, not actualized skill. Unpopular opinion time, but some of the smartest people I've meet with degrees don't have a lick of common sense, even to problem solve the simplest of tasks; people with life experience after hard setbacks seem to demonstrate better ability to navigate the world than people who rely solely on their intelligence. Often when I assess clients during BPS assessments (biopsychosocial) and through subsequent client work, I work to determine their level of insight, and understand their capacity vs their ability. As you noted, practicing on problem solving for types of problems teaches the skills of solving the problem, hence leading to greater ability to take the test, and not a true demonstration of actual skill.
Malcom Gladwell's "Outliers" I think presents a better case for achievement through having peers and mentors who teach the navigation of the social world as a greater marker for upward advancement than strictly IQ. Langan is mentioned as a case study in the book that just because you have a high IQ doesn't necessarily mean you will advance in life.
Thank you for a great read.
While we're obsessing over those on the right-hand side of the X axis, we should spare a thought for those on the left, close to the origin, those in the death-penalty states for whom the only bragging rights from a single point gained may be the right to choose your last meal.
If Eric's critique is on the money, and I think it is, then investing IQ tests with the power and precision to determine life or death, even if in only a small number of cases, is obscene.
In the continuing absence of any worthwhile definition of intelligence, I shall continue to regard IQ tests as measuring the ability of people to do IQ tests.
That's a really good point - I haven't looking to if the variance also increases the other way, but I'd assume it does. And even if it doesn't, and it's still, say, 10 point spreads, that could make someone qualify or not for the death penalty depending on what day they take the test, what test they take, etc.
Your worst essay so far. Tons of errors in this that it isn't sufficient to go over them in a comment.
But since you had enough time to read the essay and provide a haughty-sounding dismissal: Perhaps you can provide one or two examples of those numerous errors?
I’d love to read a commentary on the errors. I was also surprised...
Great article.
I'd have to give Wikipedia an IQ of about 60 or 70, same as the room temperature.
I really liked this essay, especially the line "they were people on fire with thought."
It's also really cool how I start to see threads between different pieces of yours (and other authors) here. Clearly this piece speaks to your piece on the SAT, which you mention, but also to both your essay Why we stopped making Einstein's and, in my mind, to an essay that Adam Mastroianni published recently making the argument that science is fundamentally a strong-link problem.
My background is in the humanities and I work on the line between social science and public policy today, so I am clearly biased in my own way, but I feel that the obsession with IQ speaks to a larger desire to quantify the world, which, in its myopic insistence that everything can be understood with numbers, loses the thread that the qualitative aspects of our world also matter.
Occasionally you have a really strong prior that influences the way you interface with the world on a practical level. This completely dismantled a very strong prior I had about the way IQ scores determine outcomes. My IQ was “tested” in the 120s as a kid, and I scored 2000 on my SAT w/ no prep courses or anything. I thought of myself as smart, but would never even put myself in the same universe as someone like Richard Feynman. I kind of thought this was a done deal. I think to some extent I’ve sold myself short. Obviously the point here is that IQ is just one of many components that goes in to “brain power”, and I have nowhere near the “brain power” of someone like Richard Feynman. But I think reading this will actually encourage me to test my limits a little bit harder.
Thanks Eric. Highlighted some problems of IQ testing. I am to present to my lab about the controversies of IQ testing (race x IQ = nightmare), and you have provided me some criticisms against the process itself. It also made me think of my clinical supervisor, and how she speaks about her daughter. She will make off-handed comments about her daughters IQ (145!) without adding that all her students (12 of us) use her daughter as practice for administering the WISC-V - a standardised intelligence test for children. I never have the heart to repeat back to her what she always tries to remind us about psychometrics: "Practice effects, like, they influence stuff, scores and all that. Just be careful."
There's a lot of talk about IQ these days in Substack and Podcasts but not enough about Nurture / Culture. In their Triple Pack book Amy Chua and Jed Rubenfeld talk about how "Chinese Americans are getting more bang for their intelligence buck. Chinese Americans with an IQ of (say) 103 get significantly better grades in school, scores on tests, and ultimately higher-paying jobs than do white Americans with an IQ of 103 "
That variance is not covered by IQ for sure, but it doesn’t mean it’s not genetics still.
Interesting read. Strange though, the one thing I took from it is that while IQ gives us some indicator of intelligence (which is a highly contested notion as it is) I wonder is it any better than your realization of how intelligent your colleagues are just by your own interaction with them. You didn’t need any IQ to tell you that. Is that experiential realization a better indicator than an IQ test?
Perhaps like the supreme court's pornography test: "I know it when I see it" https://en.wikipedia.org/wiki/I_know_it_when_I_see_it
i think it'd be really interesting to try to look at metrics surrounding problem-solving capabilities when answers themselves are not well-defined, as i believe the ability to tackle the unknown and to see through the fog is essential to what many consider to be "genius". with something like an IQ test or an SAT test you lose some of this critical info because as you note you can get better at test-taking by simply practicing test-taking. so it's like intelligence is correlated with IQ but there is a large dispersion on that correlation. we have to figure out what key factors drive that dispersion/if there is an alternative metric that correlates with less dispersion.
Stephen Hsu has a fairly nuanced ongoing discussion of the Study of Mathematically Precocious Youth, which tracks the life histories of a sample of very high-IQ youths. Granting all the caveats, it's still striking how much can be predicted by the results of taking an IQ test at a young age: https://infoproc.blogspot.com/search/label/smpy