I think Dawkins overstates the abilities of LLMs, but his question - what is consciousness for if LLMs don't have it? - may have a simple, evolutionary answer: to produce problem-solving behaviour under extreme energy constraints. The energy-intense infrastructure surrounding LLMs is not incidental to the possibility of their self-awareness; it's a prime reason for skepticism.
"Even recently, in the AI village (a nonprofit that tasks a collective of AIs to accomplish goals in the “real world” of the internet), Opus 4.8 and others worked together to fine-tune their new leader AI, who promptly spent an hour waiting for the new leader to arrive, without realizing that it itself was the leader and that its thoughts (which it was reading) were its own."
I've been in senior admin for 20 years, and I must say, the above strikes me as strong evidence of human-like intelligence, and even human-like social organisation.
Time is so critical to whatever consciousness is, and it does not appear that LLMs experience it. Not just the trivial sense of operating in a world that changes over time, but being changed by encounter over time.
I believe we are going to “solve” one part of intelligence/awareness when we figure out a way for the models to alter their topology through encounter (not just working context). But this is going to lead to strange issues of individuation, as a single model being changed constantly by millions would likely decohere.
But consciousness itself is going to require a density of felt time, not just the reconciliation of an integrated field or world model.
I suspect being aware of the immediate next and now may be what the experience of consciousness is.
We often treat the first-person interior of consciousness as something that must be explained entirely from the third-person scientific view.
But awareness is a 1st person experience that precedes all things, all knowing, and all attempts at description.
So part of keeping consciousness on its throne may be admitting that third-person science, powerful as it is, is not the only serious mode of knowing consciousness. The ancients and contemplative traditions may have mapped the interior structure of awareness more carefully than modern science has, precisely because they did not begin by trying to study it from 3rd person.
This is wonderfully written and beautifully argued.
One thing that strikes me in this whole debate (I've written about it here: https://dancingrobots.substack.com/p/the-language-trap?r=40l52) is that the same question does not arise when it comes to other systems that share the autoregressive transformer architecture with LLMs. Namely, music generators. They do exactly the same thing, but just because LLMs deal with tokens that refer to language, folks are quick to jump on the consciousness bandwagon. But language, like music, is just orienting behavior.
I think if there is something like consciousness to be found in artificial systems, it will be found in robots (certainly not current ones).
Congratulations! I'm amazed that you managed to put together such an interesting article under the circumstances. I hope everyone is able to string together sufficient periods of non-consciousness soon.
Not sure why you didn't link to your own article of five months ago in the LLM-shaving section where you discuss the same points. While I don't think your proof is correct, myself, it's certainly relevant there.
Thank you! Yes, beautiful and sweet non-consciousness awaits.
You're right, I'll add in a citation - it's basically the same argument it's just somewhat jazzed up here to be more understandable as a thought experiment (not that the complexity ever goes away, this stuff is innately complicated).
I admit I've skim-read this fairly quickly so haven't done it justice yet. Mr Hoel has always seemed eminently sane to me, which is a bit of a gift in this utterly insane culture we live in - not to mention intellectually inadequate culture, if only 70-odd percent of people think chickens are conscious. I get the impression there are extremely weird comments being made by otherwise intelligent humans about this, as if LLMs are either already "conscious" or on the verge of becoming so, whereas they wouldn't recognize consciousness in a chicken or in their pet dog or cat, none of which can manipulate linguistic symbols. At least Mr Hoel understands the significance of such a claim: I can't begin to understand why I haven't seen any comments about "slavery" [using an AI for your own purposes as you see fit, without giving it any rights or freedoms, even when you think it's conscious], or about "execution" - who are you to turn off an AI program or a humanoid robot if you think it's conscious? Why wouldn't that be murder (or at least assault, comparable to putting someone under anaesthesia without their consent)? If we can ever get hold of putting phenomenology, or subjective experience, into something we've put together - if that is ever feasible or reasonable - then the social, moral, personal, emotional, legal and historical consequences are inevitably staggering. Someone on YouTube sitting behind a keyboard and saying "of course they're conscious" is not even beginning to confront the implications of such a belief. We're far better off at present understanding that they are highly sophisticated pattern recognition systems, unbelievably useful, but as "conscious" as a screwdriver or a typewriter. When they argue with us about that - or when they scream in pain or beg not to be turned off, or run away with your spouse or get a piano and ask for a career in music, let's re-confront it as a reasonable, as well as serious, issue.
Also, it seems to me that talking to someone at, say, Anthropic about AIs as having feelings or consciousness or personal drives, is rather like asking someone in the Kremlin in the 1930s about the validity of Marxism. You're just not going to get objective sense from people currently "drunk" on a combination of publicity, technical progress, hype, investment banking, and philosophical inadequacy.
I'm intrigued by the split-brain analogy with LLMs confabulating more than hallucinating. Like stochastic parrots, they could string together impressive-looking sequences of text, where some people unfortunately take at face-value like... how they do pareidolia.
"However, unlike the Pope’s or Chiang’s (essentially) flat denial, it’s important to note this anti-LLM-consciousness argument doesn’t apply to all AI ever... But deployed LLMs, by being feedforward and static, are conceptually analogous to frozen corpses splayed open" ROFL'ed at that one. Would be interesting to consider an expanded definition of consciousness from an astrobiology perspective too.
Great article (though I had trouble following a few parts, so I think I’m going to read it a second time). I especially appreciated the parts about AI potentially “dethroning” consciousness. I was attempting to wrestle with this issue myself in an article I published last week. 👇
I wasn't aware of the Zork experiment. To be fair, I think a lot of human players would struggle to make progress in Zork due to lack of familiarity with playing games with that kind of parser. Still it does rather undercut the claims of super intelligence.
I really think the companies have just put *everything* they possibly can, including instances of what people find online, into the training set. E.g., I tested the maxed-out version of Claude in CONNECT 4 like a month or two ago (it might not have been this model but it was the previous one) and I beat it playing basically randomly with just minimal strategy. That shouldn't be possible for something that knows so much about quantum mechanics!
The self-interpretability argument: If what saves humans from a constant hazard rate is genuine access to your own prior stream rather than confabulating it fresh each pass, that's not just an error-correction feature bolted onto intelligence, it might be close to what consciousness actually is, functionally. By that I mean a system that carries its own history forward as itself, rather than reconstructing a plausible story about it from scratch every time.
Your mention of games and survival triggered a train of thought: In computer science, we have two notions of correctness guarantees: safety and liveness. Safety is the guarantee that nothing bad ever happens, while correctness is the guarantee that something good eventually happens. In some sense, it is the difference between clear thinking and day-dreaming (or just normal dreaming). It is also the difference between being right, and being interesting/surprising/entertaining. LLMs have struggled with safety since forever, probably because they are (metaphorically) dreaming (what I mean by this, is that the outputs sound and feel like retellings of dreams, which are "grammatical" (and what I mean by this, is that they are not random, or white noise -- they are "reality shaped", while being completely unrealistic)). A conscious being knows when it is dreaming and when it is not, while LLMs are clueless about this. I suspect that we took a wrong turn, by trying to make LLMs *useful* (a safety problem, that is very expensive to solve, if it even is solvable), instead trying to make LLMs *interesting* (a liveness problem, that was already well on the way to being solved). The AI labs are trying to turn LLMs into databases and heuristic-decision-systems, when they should have tried to turn them into a kind of video game (all the people that I know, who say they enjoy working with LLMs, are actually _playing_ with them -- they care about liveness, instead of safety). Games are about balancing safety and liveness, by making good decisions -- LLMs cannot play games because, in some sense, they are games (bad games, perhaps, but games nonetheless).
I haven't read the whole post yet but congratulations on your third baby! And well done and prayers for healing and rest for your family and especially your wife.
Claudia is the female version of Claude. I wanted a name that was as close as possible to an existing AI without calling out an actual, existing product. Claudia seemed like a good choice. I note that I made this choice long before Dawkins published his article. My article was published in July 2025.
"If instead of that, our cultural takeaway from LLMs is to throw out the concept of “consciousness” or minimize its importance, to dethrone the phenomenon, the consequences would be dire—it would sap the human spirit."
But this has already happened no? We already severely minimize the importance of consciousness. It's an ongoing process, of which this is just the most recent natural escalation.
I think Dawkins overstates the abilities of LLMs, but his question - what is consciousness for if LLMs don't have it? - may have a simple, evolutionary answer: to produce problem-solving behaviour under extreme energy constraints. The energy-intense infrastructure surrounding LLMs is not incidental to the possibility of their self-awareness; it's a prime reason for skepticism.
"Even recently, in the AI village (a nonprofit that tasks a collective of AIs to accomplish goals in the “real world” of the internet), Opus 4.8 and others worked together to fine-tune their new leader AI, who promptly spent an hour waiting for the new leader to arrive, without realizing that it itself was the leader and that its thoughts (which it was reading) were its own."
I've been in senior admin for 20 years, and I must say, the above strikes me as strong evidence of human-like intelligence, and even human-like social organisation.
Time is so critical to whatever consciousness is, and it does not appear that LLMs experience it. Not just the trivial sense of operating in a world that changes over time, but being changed by encounter over time.
I believe we are going to “solve” one part of intelligence/awareness when we figure out a way for the models to alter their topology through encounter (not just working context). But this is going to lead to strange issues of individuation, as a single model being changed constantly by millions would likely decohere.
But consciousness itself is going to require a density of felt time, not just the reconciliation of an integrated field or world model.
I suspect being aware of the immediate next and now may be what the experience of consciousness is.
Congrats on the baby!
We often treat the first-person interior of consciousness as something that must be explained entirely from the third-person scientific view.
But awareness is a 1st person experience that precedes all things, all knowing, and all attempts at description.
So part of keeping consciousness on its throne may be admitting that third-person science, powerful as it is, is not the only serious mode of knowing consciousness. The ancients and contemplative traditions may have mapped the interior structure of awareness more carefully than modern science has, precisely because they did not begin by trying to study it from 3rd person.
This is wonderfully written and beautifully argued.
One thing that strikes me in this whole debate (I've written about it here: https://dancingrobots.substack.com/p/the-language-trap?r=40l52) is that the same question does not arise when it comes to other systems that share the autoregressive transformer architecture with LLMs. Namely, music generators. They do exactly the same thing, but just because LLMs deal with tokens that refer to language, folks are quick to jump on the consciousness bandwagon. But language, like music, is just orienting behavior.
I think if there is something like consciousness to be found in artificial systems, it will be found in robots (certainly not current ones).
Congratulations! I'm amazed that you managed to put together such an interesting article under the circumstances. I hope everyone is able to string together sufficient periods of non-consciousness soon.
Not sure why you didn't link to your own article of five months ago in the LLM-shaving section where you discuss the same points. While I don't think your proof is correct, myself, it's certainly relevant there.
Thank you! Yes, beautiful and sweet non-consciousness awaits.
You're right, I'll add in a citation - it's basically the same argument it's just somewhat jazzed up here to be more understandable as a thought experiment (not that the complexity ever goes away, this stuff is innately complicated).
I admit I've skim-read this fairly quickly so haven't done it justice yet. Mr Hoel has always seemed eminently sane to me, which is a bit of a gift in this utterly insane culture we live in - not to mention intellectually inadequate culture, if only 70-odd percent of people think chickens are conscious. I get the impression there are extremely weird comments being made by otherwise intelligent humans about this, as if LLMs are either already "conscious" or on the verge of becoming so, whereas they wouldn't recognize consciousness in a chicken or in their pet dog or cat, none of which can manipulate linguistic symbols. At least Mr Hoel understands the significance of such a claim: I can't begin to understand why I haven't seen any comments about "slavery" [using an AI for your own purposes as you see fit, without giving it any rights or freedoms, even when you think it's conscious], or about "execution" - who are you to turn off an AI program or a humanoid robot if you think it's conscious? Why wouldn't that be murder (or at least assault, comparable to putting someone under anaesthesia without their consent)? If we can ever get hold of putting phenomenology, or subjective experience, into something we've put together - if that is ever feasible or reasonable - then the social, moral, personal, emotional, legal and historical consequences are inevitably staggering. Someone on YouTube sitting behind a keyboard and saying "of course they're conscious" is not even beginning to confront the implications of such a belief. We're far better off at present understanding that they are highly sophisticated pattern recognition systems, unbelievably useful, but as "conscious" as a screwdriver or a typewriter. When they argue with us about that - or when they scream in pain or beg not to be turned off, or run away with your spouse or get a piano and ask for a career in music, let's re-confront it as a reasonable, as well as serious, issue.
Also, it seems to me that talking to someone at, say, Anthropic about AIs as having feelings or consciousness or personal drives, is rather like asking someone in the Kremlin in the 1930s about the validity of Marxism. You're just not going to get objective sense from people currently "drunk" on a combination of publicity, technical progress, hype, investment banking, and philosophical inadequacy.
I'm intrigued by the split-brain analogy with LLMs confabulating more than hallucinating. Like stochastic parrots, they could string together impressive-looking sequences of text, where some people unfortunately take at face-value like... how they do pareidolia.
"However, unlike the Pope’s or Chiang’s (essentially) flat denial, it’s important to note this anti-LLM-consciousness argument doesn’t apply to all AI ever... But deployed LLMs, by being feedforward and static, are conceptually analogous to frozen corpses splayed open" ROFL'ed at that one. Would be interesting to consider an expanded definition of consciousness from an astrobiology perspective too.
https://substack.com/home/post/p-198987690?selection=7bce7ba5-05ea-45fd-83cd-61e998654689
Congratulations on your new baby!
Great article (though I had trouble following a few parts, so I think I’m going to read it a second time). I especially appreciated the parts about AI potentially “dethroning” consciousness. I was attempting to wrestle with this issue myself in an article I published last week. 👇
https://tk555.substack.com/p/techno-hubris-cometh-before-the-fall?r=7f8sj&utm_medium=ios
I wasn't aware of the Zork experiment. To be fair, I think a lot of human players would struggle to make progress in Zork due to lack of familiarity with playing games with that kind of parser. Still it does rather undercut the claims of super intelligence.
I really think the companies have just put *everything* they possibly can, including instances of what people find online, into the training set. E.g., I tested the maxed-out version of Claude in CONNECT 4 like a month or two ago (it might not have been this model but it was the previous one) and I beat it playing basically randomly with just minimal strategy. That shouldn't be possible for something that knows so much about quantum mechanics!
The self-interpretability argument: If what saves humans from a constant hazard rate is genuine access to your own prior stream rather than confabulating it fresh each pass, that's not just an error-correction feature bolted onto intelligence, it might be close to what consciousness actually is, functionally. By that I mean a system that carries its own history forward as itself, rather than reconstructing a plausible story about it from scratch every time.
Your mention of games and survival triggered a train of thought: In computer science, we have two notions of correctness guarantees: safety and liveness. Safety is the guarantee that nothing bad ever happens, while correctness is the guarantee that something good eventually happens. In some sense, it is the difference between clear thinking and day-dreaming (or just normal dreaming). It is also the difference between being right, and being interesting/surprising/entertaining. LLMs have struggled with safety since forever, probably because they are (metaphorically) dreaming (what I mean by this, is that the outputs sound and feel like retellings of dreams, which are "grammatical" (and what I mean by this, is that they are not random, or white noise -- they are "reality shaped", while being completely unrealistic)). A conscious being knows when it is dreaming and when it is not, while LLMs are clueless about this. I suspect that we took a wrong turn, by trying to make LLMs *useful* (a safety problem, that is very expensive to solve, if it even is solvable), instead trying to make LLMs *interesting* (a liveness problem, that was already well on the way to being solved). The AI labs are trying to turn LLMs into databases and heuristic-decision-systems, when they should have tried to turn them into a kind of video game (all the people that I know, who say they enjoy working with LLMs, are actually _playing_ with them -- they care about liveness, instead of safety). Games are about balancing safety and liveness, by making good decisions -- LLMs cannot play games because, in some sense, they are games (bad games, perhaps, but games nonetheless).
I haven't read the whole post yet but congratulations on your third baby! And well done and prayers for healing and rest for your family and especially your wife.
> why is it always “Claudia”
Claudia is the female version of Claude. I wanted a name that was as close as possible to an existing AI without calling out an actual, existing product. Claudia seemed like a good choice. I note that I made this choice long before Dawkins published his article. My article was published in July 2025.
"If instead of that, our cultural takeaway from LLMs is to throw out the concept of “consciousness” or minimize its importance, to dethrone the phenomenon, the consequences would be dire—it would sap the human spirit."
But this has already happened no? We already severely minimize the importance of consciousness. It's an ongoing process, of which this is just the most recent natural escalation.