I think it was Charles Murray who compared IQ scores to the weight of offensive lineman - there's a minimum threshold that you need to meet to really compete at the higher levels, but once you pass that threshold, more does not correlate at all with higher performance. Winning at those higher levels requires those things that are difficult or impossible to measure, the intangibles, that being "on fire with thought"... or to carry on the football analogy, being on fire with pushing those other dudes out of the way.
I appreciate your touching on the middle ground between "IQ tests are gospel" and "IQ is made up." It seems pretty clear to me that: a) people differ in their intellectual abilities in ways that are at least somewhat innate (though it's also possible to excel or lag in some areas but not others); b) it's not entirely innate; and c) it's hard to have an entirely unbiased test intellectual ability, let alone tease out innate from acquired ability, particularly in a given person. However, there seem to be a lot of people who like to use (b) and (c) to "prove" (a) isn't true.
I agree with the main point of this post (attempts to judge famous people's IQs are garbage, IQ of 160 is basically unmeasurable), but I think along the way it gets some things importantly wrong.
1. It uses the word IQ in a way that equivocates between "score on an IQ test" and "g, the thing IQ tests measure". Many of its points don't make sense once you remove that equivocation. For example, it's not true that nobody's IQ is 160. Certainly some people get scores of 160 on IQ tests. And some people are four standard deviations from the mean on g. It's just that they're probably not the same people, or we can't say with confidence that they are.
2. Likewise, when you say "IQ is changeable", you mean "score on IQ tests". This is no more interesting than saying that height "is changeable" because you can stand on your tiptoes when they're measuring you, or wear platform shoes. g doesn't seem to be changeable, or at least it would take much more evidence to convince me that it was.
3. The part about benefits of high IQ vanishing past a certain point is probably wrong. The study you cite has some issues and is contradicted by other studies; see Link 23 at https://astralcodexten.substack.com/p/links-for-march-2023 for directions to some other places that can explain the problems better than I can.
4. I don't have an airtight retort to the Einstein story except to ask whether you think it would be fair to say Einstein "wasn't very good at math". "Good at math" seems to be the sort of thing which is heavily correlated with math grades and math test scores, in the same way that mathematical IQ is correlated with math grades and math test scores. So I see only two options. First, Einstein wasn't very good at math (this doesn't seem impossible to me, maybe he just worked really hard, got enough math to do physics at all, and was truly exceptional in some other area like creativity). Second, he was good at math, but he found his classes boring or oppressive, didn't try very hard, and so his grades don't reflect his talent very well. I think switching from the mystical-sounding word "IQ" to the normal-sounding words "good at math" help prime our intuition here and make the second possibility seem pretty likely. See https://www.openculture.com/2020/04/albert-einsteins-grades-a-fascinating-look-at-his-report-cards.html for more evidence that this is true.
That having been said, I think Kasparov had a real IQ test and it was 135. It wouldn't surprise me if top-in-the-world-at-something geniuses often had about IQ 135 because their skill is only modestly correlated with IQ. My guess is that some geniuses will be around 135 (because they're using something like creativity to succeed) and others will be higher (because they're using raw intelligence).
Hopefully someone smarter than me will come along soon to address some of the other points.
Well, this was triggering. I test well, but perform badly. This has been a curse. Nobody should ever give a child an IQ test which will follow her like a stench throughout her schooling, provoking standoffs between parents and teachers blaming one another for her failure to reach her full potential. And don’t get me started on the smug Gould; for awhile, he was everywhere I went, and you’d have to either leave immediately or be stuck like a hostage, listening to him for hours. I’m still not sure if people really thought he was a genius, or if his agent was the actual genius.
The issue of motivation is salient to me. I'm not surprised that Einstein did poorly or unexceptionally at school. Maybe he just wasn't interested. I did poorly in most of my teen years -- lots of D and C-s, with an occasional B and one high score when biology moved from insects (boring!) to humans (interesting!). I had to re-do my last year (O-level time in England). However, once I became fascinated with philosophy, politics, and economics, I suddenly went to the top of my classes, was allowed to take the Oxbridge exams, got it, and ended up with a doctorate. Interest and motivation was *extremely* important. Some people seem different and can make themselves study things they find boring. Not me. Maybe not Einstein.
I thought this was an excellent essay, and very well outlines the problems with IQ testing. I had to take a class in assessment for my graduate degree, and understanding the pros and cons of each is helpful. Unfortunately, people utilize these tests as a marker of ability or skill, and it seems fair to say that IQ is more of a test of potential, not actualized skill. Unpopular opinion time, but some of the smartest people I've meet with degrees don't have a lick of common sense, even to problem solve the simplest of tasks; people with life experience after hard setbacks seem to demonstrate better ability to navigate the world than people who rely solely on their intelligence. Often when I assess clients during BPS assessments (biopsychosocial) and through subsequent client work, I work to determine their level of insight, and understand their capacity vs their ability. As you noted, practicing on problem solving for types of problems teaches the skills of solving the problem, hence leading to greater ability to take the test, and not a true demonstration of actual skill.
Malcom Gladwell's "Outliers" I think presents a better case for achievement through having peers and mentors who teach the navigation of the social world as a greater marker for upward advancement than strictly IQ. Langan is mentioned as a case study in the book that just because you have a high IQ doesn't necessarily mean you will advance in life.
While we're obsessing over those on the right-hand side of the X axis, we should spare a thought for those on the left, close to the origin, those in the death-penalty states for whom the only bragging rights from a single point gained may be the right to choose your last meal.
If Eric's critique is on the money, and I think it is, then investing IQ tests with the power and precision to determine life or death, even if in only a small number of cases, is obscene.
In the continuing absence of any worthwhile definition of intelligence, I shall continue to regard IQ tests as measuring the ability of people to do IQ tests.
I really liked this essay, especially the line "they were people on fire with thought."
It's also really cool how I start to see threads between different pieces of yours (and other authors) here. Clearly this piece speaks to your piece on the SAT, which you mention, but also to both your essay Why we stopped making Einstein's and, in my mind, to an essay that Adam Mastroianni published recently making the argument that science is fundamentally a strong-link problem.
My background is in the humanities and I work on the line between social science and public policy today, so I am clearly biased in my own way, but I feel that the obsession with IQ speaks to a larger desire to quantify the world, which, in its myopic insistence that everything can be understood with numbers, loses the thread that the qualitative aspects of our world also matter.
Occasionally you have a really strong prior that influences the way you interface with the world on a practical level. This completely dismantled a very strong prior I had about the way IQ scores determine outcomes. My IQ was “tested” in the 120s as a kid, and I scored 2000 on my SAT w/ no prep courses or anything. I thought of myself as smart, but would never even put myself in the same universe as someone like Richard Feynman. I kind of thought this was a done deal. I think to some extent I’ve sold myself short. Obviously the point here is that IQ is just one of many components that goes in to “brain power”, and I have nowhere near the “brain power” of someone like Richard Feynman. But I think reading this will actually encourage me to test my limits a little bit harder.
Thanks Eric. Highlighted some problems of IQ testing. I am to present to my lab about the controversies of IQ testing (race x IQ = nightmare), and you have provided me some criticisms against the process itself. It also made me think of my clinical supervisor, and how she speaks about her daughter. She will make off-handed comments about her daughters IQ (145!) without adding that all her students (12 of us) use her daughter as practice for administering the WISC-V - a standardised intelligence test for children. I never have the heart to repeat back to her what she always tries to remind us about psychometrics: "Practice effects, like, they influence stuff, scores and all that. Just be careful."
There's a lot of talk about IQ these days in Substack and Podcasts but not enough about Nurture / Culture. In their Triple Pack book Amy Chua and Jed Rubenfeld talk about how "Chinese Americans are getting more bang for their intelligence buck. Chinese Americans with an IQ of (say) 103 get significantly better grades in school, scores on tests, and ultimately higher-paying jobs than do white Americans with an IQ of 103 "
Interesting read. Strange though, the one thing I took from it is that while IQ gives us some indicator of intelligence (which is a highly contested notion as it is) I wonder is it any better than your realization of how intelligent your colleagues are just by your own interaction with them. You didn’t need any IQ to tell you that. Is that experiential realization a better indicator than an IQ test?
i think it'd be really interesting to try to look at metrics surrounding problem-solving capabilities when answers themselves are not well-defined, as i believe the ability to tackle the unknown and to see through the fog is essential to what many consider to be "genius". with something like an IQ test or an SAT test you lose some of this critical info because as you note you can get better at test-taking by simply practicing test-taking. so it's like intelligence is correlated with IQ but there is a large dispersion on that correlation. we have to figure out what key factors drive that dispersion/if there is an alternative metric that correlates with less dispersion.
Stephen Hsu has a fairly nuanced ongoing discussion of the Study of Mathematically Precocious Youth, which tracks the life histories of a sample of very high-IQ youths. Granting all the caveats, it's still striking how much can be predicted by the results of taking an IQ test at a young age: https://infoproc.blogspot.com/search/label/smpy
I think it was Charles Murray who compared IQ scores to the weight of offensive lineman - there's a minimum threshold that you need to meet to really compete at the higher levels, but once you pass that threshold, more does not correlate at all with higher performance. Winning at those higher levels requires those things that are difficult or impossible to measure, the intangibles, that being "on fire with thought"... or to carry on the football analogy, being on fire with pushing those other dudes out of the way.
I appreciate your touching on the middle ground between "IQ tests are gospel" and "IQ is made up." It seems pretty clear to me that: a) people differ in their intellectual abilities in ways that are at least somewhat innate (though it's also possible to excel or lag in some areas but not others); b) it's not entirely innate; and c) it's hard to have an entirely unbiased test intellectual ability, let alone tease out innate from acquired ability, particularly in a given person. However, there seem to be a lot of people who like to use (b) and (c) to "prove" (a) isn't true.
I agree with the main point of this post (attempts to judge famous people's IQs are garbage, IQ of 160 is basically unmeasurable), but I think along the way it gets some things importantly wrong.
1. It uses the word IQ in a way that equivocates between "score on an IQ test" and "g, the thing IQ tests measure". Many of its points don't make sense once you remove that equivocation. For example, it's not true that nobody's IQ is 160. Certainly some people get scores of 160 on IQ tests. And some people are four standard deviations from the mean on g. It's just that they're probably not the same people, or we can't say with confidence that they are.
2. Likewise, when you say "IQ is changeable", you mean "score on IQ tests". This is no more interesting than saying that height "is changeable" because you can stand on your tiptoes when they're measuring you, or wear platform shoes. g doesn't seem to be changeable, or at least it would take much more evidence to convince me that it was.
3. The part about benefits of high IQ vanishing past a certain point is probably wrong. The study you cite has some issues and is contradicted by other studies; see Link 23 at https://astralcodexten.substack.com/p/links-for-march-2023 for directions to some other places that can explain the problems better than I can.
4. I don't have an airtight retort to the Einstein story except to ask whether you think it would be fair to say Einstein "wasn't very good at math". "Good at math" seems to be the sort of thing which is heavily correlated with math grades and math test scores, in the same way that mathematical IQ is correlated with math grades and math test scores. So I see only two options. First, Einstein wasn't very good at math (this doesn't seem impossible to me, maybe he just worked really hard, got enough math to do physics at all, and was truly exceptional in some other area like creativity). Second, he was good at math, but he found his classes boring or oppressive, didn't try very hard, and so his grades don't reflect his talent very well. I think switching from the mystical-sounding word "IQ" to the normal-sounding words "good at math" help prime our intuition here and make the second possibility seem pretty likely. See https://www.openculture.com/2020/04/albert-einsteins-grades-a-fascinating-look-at-his-report-cards.html for more evidence that this is true.
That having been said, I think Kasparov had a real IQ test and it was 135. It wouldn't surprise me if top-in-the-world-at-something geniuses often had about IQ 135 because their skill is only modestly correlated with IQ. My guess is that some geniuses will be around 135 (because they're using something like creativity to succeed) and others will be higher (because they're using raw intelligence).
Hopefully someone smarter than me will come along soon to address some of the other points.
Well, this was triggering. I test well, but perform badly. This has been a curse. Nobody should ever give a child an IQ test which will follow her like a stench throughout her schooling, provoking standoffs between parents and teachers blaming one another for her failure to reach her full potential. And don’t get me started on the smug Gould; for awhile, he was everywhere I went, and you’d have to either leave immediately or be stuck like a hostage, listening to him for hours. I’m still not sure if people really thought he was a genius, or if his agent was the actual genius.
The issue of motivation is salient to me. I'm not surprised that Einstein did poorly or unexceptionally at school. Maybe he just wasn't interested. I did poorly in most of my teen years -- lots of D and C-s, with an occasional B and one high score when biology moved from insects (boring!) to humans (interesting!). I had to re-do my last year (O-level time in England). However, once I became fascinated with philosophy, politics, and economics, I suddenly went to the top of my classes, was allowed to take the Oxbridge exams, got it, and ended up with a doctorate. Interest and motivation was *extremely* important. Some people seem different and can make themselves study things they find boring. Not me. Maybe not Einstein.
I thought this was an excellent essay, and very well outlines the problems with IQ testing. I had to take a class in assessment for my graduate degree, and understanding the pros and cons of each is helpful. Unfortunately, people utilize these tests as a marker of ability or skill, and it seems fair to say that IQ is more of a test of potential, not actualized skill. Unpopular opinion time, but some of the smartest people I've meet with degrees don't have a lick of common sense, even to problem solve the simplest of tasks; people with life experience after hard setbacks seem to demonstrate better ability to navigate the world than people who rely solely on their intelligence. Often when I assess clients during BPS assessments (biopsychosocial) and through subsequent client work, I work to determine their level of insight, and understand their capacity vs their ability. As you noted, practicing on problem solving for types of problems teaches the skills of solving the problem, hence leading to greater ability to take the test, and not a true demonstration of actual skill.
Malcom Gladwell's "Outliers" I think presents a better case for achievement through having peers and mentors who teach the navigation of the social world as a greater marker for upward advancement than strictly IQ. Langan is mentioned as a case study in the book that just because you have a high IQ doesn't necessarily mean you will advance in life.
Thank you for a great read.
While we're obsessing over those on the right-hand side of the X axis, we should spare a thought for those on the left, close to the origin, those in the death-penalty states for whom the only bragging rights from a single point gained may be the right to choose your last meal.
If Eric's critique is on the money, and I think it is, then investing IQ tests with the power and precision to determine life or death, even if in only a small number of cases, is obscene.
In the continuing absence of any worthwhile definition of intelligence, I shall continue to regard IQ tests as measuring the ability of people to do IQ tests.
Great article.
I'd have to give Wikipedia an IQ of about 60 or 70, same as the room temperature.
Your worst essay so far. Tons of errors in this that it isn't sufficient to go over them in a comment.
I really liked this essay, especially the line "they were people on fire with thought."
It's also really cool how I start to see threads between different pieces of yours (and other authors) here. Clearly this piece speaks to your piece on the SAT, which you mention, but also to both your essay Why we stopped making Einstein's and, in my mind, to an essay that Adam Mastroianni published recently making the argument that science is fundamentally a strong-link problem.
My background is in the humanities and I work on the line between social science and public policy today, so I am clearly biased in my own way, but I feel that the obsession with IQ speaks to a larger desire to quantify the world, which, in its myopic insistence that everything can be understood with numbers, loses the thread that the qualitative aspects of our world also matter.
Occasionally you have a really strong prior that influences the way you interface with the world on a practical level. This completely dismantled a very strong prior I had about the way IQ scores determine outcomes. My IQ was “tested” in the 120s as a kid, and I scored 2000 on my SAT w/ no prep courses or anything. I thought of myself as smart, but would never even put myself in the same universe as someone like Richard Feynman. I kind of thought this was a done deal. I think to some extent I’ve sold myself short. Obviously the point here is that IQ is just one of many components that goes in to “brain power”, and I have nowhere near the “brain power” of someone like Richard Feynman. But I think reading this will actually encourage me to test my limits a little bit harder.
Thanks Eric. Highlighted some problems of IQ testing. I am to present to my lab about the controversies of IQ testing (race x IQ = nightmare), and you have provided me some criticisms against the process itself. It also made me think of my clinical supervisor, and how she speaks about her daughter. She will make off-handed comments about her daughters IQ (145!) without adding that all her students (12 of us) use her daughter as practice for administering the WISC-V - a standardised intelligence test for children. I never have the heart to repeat back to her what she always tries to remind us about psychometrics: "Practice effects, like, they influence stuff, scores and all that. Just be careful."
There's a lot of talk about IQ these days in Substack and Podcasts but not enough about Nurture / Culture. In their Triple Pack book Amy Chua and Jed Rubenfeld talk about how "Chinese Americans are getting more bang for their intelligence buck. Chinese Americans with an IQ of (say) 103 get significantly better grades in school, scores on tests, and ultimately higher-paying jobs than do white Americans with an IQ of 103 "
Interesting read. Strange though, the one thing I took from it is that while IQ gives us some indicator of intelligence (which is a highly contested notion as it is) I wonder is it any better than your realization of how intelligent your colleagues are just by your own interaction with them. You didn’t need any IQ to tell you that. Is that experiential realization a better indicator than an IQ test?
i think it'd be really interesting to try to look at metrics surrounding problem-solving capabilities when answers themselves are not well-defined, as i believe the ability to tackle the unknown and to see through the fog is essential to what many consider to be "genius". with something like an IQ test or an SAT test you lose some of this critical info because as you note you can get better at test-taking by simply practicing test-taking. so it's like intelligence is correlated with IQ but there is a large dispersion on that correlation. we have to figure out what key factors drive that dispersion/if there is an alternative metric that correlates with less dispersion.
Stephen Hsu has a fairly nuanced ongoing discussion of the Study of Mathematically Precocious Youth, which tracks the life histories of a sample of very high-IQ youths. Granting all the caveats, it's still striking how much can be predicted by the results of taking an IQ test at a young age: https://infoproc.blogspot.com/search/label/smpy