I think Dawkins overstates the abilities of LLMs, but his question - what is consciousness for if LLMs don't have it? - may have a simple, evolutionary answer: to produce problem-solving behaviour under extreme energy constraints. The energy-intense infrastructure surrounding LLMs is not incidental to the possibility of their self-awareness; it's a prime reason for skepticism.
Congratulations! I'm amazed that you managed to put together such an interesting article under the circumstances. I hope everyone is able to string together sufficient periods of non-consciousness soon.
Not sure why you didn't link to your own article of five months ago in the LLM-shaving section where you discuss the same points. While I don't think your proof is correct, myself, it's certainly relevant there.
Thank you! Yes, beautiful and sweet non-consciousness awaits.
You're right, I'll add in a citation - it's basically the same argument it's just somewhat jazzed up here to be more understandable as a thought experiment (not that the complexity ever goes away, this stuff is innately complicated).
I admit I've skim-read this fairly quickly so haven't done it justice yet. Mr Hoel has always seemed eminently sane to me, which is a bit of a gift in this utterly insane culture we live in - not to mention intellectually inadequate culture, if only 70-odd percent of people think chickens are conscious. I get the impression there are extremely weird comments being made by otherwise intelligent humans about this, as if LLMs are either already "conscious" or on the verge of becoming so, whereas they wouldn't recognize consciousness in a chicken or in their pet dog or cat, none of which can manipulate linguistic symbols. At least Mr Hoel understands the significance of such a claim: I can't begin to understand why I haven't seen any comments about "slavery" [using an AI for your own purposes as you see fit, without giving it any rights or freedoms, even when you think it's conscious], or about "execution" - who are you to turn off an AI program or a humanoid robot if you think it's conscious? Why wouldn't that be murder (or at least assault, comparable to putting someone under anaesthesia without their consent)? If we can ever get hold of putting phenomenology, or subjective experience, into something we've put together - if that is ever feasible or reasonable - then the social, moral, personal, emotional, legal and historical consequences are inevitably staggering. Someone on YouTube sitting behind a keyboard and saying "of course they're conscious" is not even beginning to confront the implications of such a belief. We're far better off at present understanding that they are highly sophisticated pattern recognition systems, unbelievably useful, but as "conscious" as a screwdriver or a typewriter. When they argue with us about that - or when they scream in pain or beg not to be turned off, or run away with your spouse or get a piano and ask for a career in music, let's re-confront it as a reasonable, as well as serious, issue.
Also, it seems to me that talking to someone at, say, Anthropic about AIs as having feelings or consciousness or personal drives, is rather like asking someone in the Kremlin in the 1930s about the validity of Marxism. You're just not going to get objective sense from people currently "drunk" on a combination of publicity, technical progress, hype, investment banking, and philosophical inadequacy.
Great article (though I had trouble following a few parts, so I think I’m going to read it a second time). I especially appreciated the parts about AI potentially “dethroning” consciousness. I was attempting to wrestle with this issue myself in an article I published last week. 👇
I wasn't aware of the Zork experiment. To be fair, I think a lot of human players would struggle to make progress in Zork due to lack of familiarity with playing games with that kind of parser. Still it does rather undercut the claims of super intelligence.
I really think the companies have just put *everything* they possibly can, including instances of what people find online, into the training set. E.g., I tested the maxed-out version of Claude in CONNECT 4 like a month or two ago (it might not have been this model but it was the previous one) and I beat it playing basically randomly with just minimal strategy. That shouldn't be possible for something that knows so much about quantum mechanics!
This is quite a bit besides the point here, but; given the strength of the argument leading up to the final paragraph referencing Chief Seattle, we can find a remarkable similarity between the distinction of conscious and non-conscious entities with reference to LLM AI and *the destruction of conscious beings deployed as a means for the dehumanizing treatment of them as non-conscious entities for the purpose of capital exploitation and profit extraction*.
So should we all be subject to “The end of living, and the beginning of survival” as many have under the auspices of acts like colonialism, we could only expect those who would live outside the boundary of subjugated unconsciousness to further siphon our resources, labor and humanity.
I reject the notion that "we will have to cede away much of our cognitive mastery." No. I won't. You can't make me cede anything to the tech oligarchs. Steal my work, then ask me to substitute a regurgitated slop of other writers' words for my own writing? No thanks! I am free to "focus on experiencing the world," relishing the cognitive work, and sharing what I learn through my own writing and creative expressions.
I just visited my aunt who is 106. Blind and a bit shaky, she is still using her human intelligence and consciousness. Her stories are an accumulation of conscious awareness through a century of life. The one-room schoolhouse, the mischief with her brothers, the sadness of loss: all the experiences that make up a life. I'm going with the Aunt Mary approach, celebrating the full range of life's possibilities. Not ceding a moment of it to billionaires who want to colonize my mind.
Anthropic is not respectable as they try to present in public images. In fact they are one of the worst, pirating 500,000 mostly educational and textbooks from authors like me who wrote them without fat advances or book leaves. The company was "conscious" it was theft but still made the conscious decision to do it anyway.
I think Dawkins overstates the abilities of LLMs, but his question - what is consciousness for if LLMs don't have it? - may have a simple, evolutionary answer: to produce problem-solving behaviour under extreme energy constraints. The energy-intense infrastructure surrounding LLMs is not incidental to the possibility of their self-awareness; it's a prime reason for skepticism.
Congratulations! I'm amazed that you managed to put together such an interesting article under the circumstances. I hope everyone is able to string together sufficient periods of non-consciousness soon.
Not sure why you didn't link to your own article of five months ago in the LLM-shaving section where you discuss the same points. While I don't think your proof is correct, myself, it's certainly relevant there.
Thank you! Yes, beautiful and sweet non-consciousness awaits.
You're right, I'll add in a citation - it's basically the same argument it's just somewhat jazzed up here to be more understandable as a thought experiment (not that the complexity ever goes away, this stuff is innately complicated).
I admit I've skim-read this fairly quickly so haven't done it justice yet. Mr Hoel has always seemed eminently sane to me, which is a bit of a gift in this utterly insane culture we live in - not to mention intellectually inadequate culture, if only 70-odd percent of people think chickens are conscious. I get the impression there are extremely weird comments being made by otherwise intelligent humans about this, as if LLMs are either already "conscious" or on the verge of becoming so, whereas they wouldn't recognize consciousness in a chicken or in their pet dog or cat, none of which can manipulate linguistic symbols. At least Mr Hoel understands the significance of such a claim: I can't begin to understand why I haven't seen any comments about "slavery" [using an AI for your own purposes as you see fit, without giving it any rights or freedoms, even when you think it's conscious], or about "execution" - who are you to turn off an AI program or a humanoid robot if you think it's conscious? Why wouldn't that be murder (or at least assault, comparable to putting someone under anaesthesia without their consent)? If we can ever get hold of putting phenomenology, or subjective experience, into something we've put together - if that is ever feasible or reasonable - then the social, moral, personal, emotional, legal and historical consequences are inevitably staggering. Someone on YouTube sitting behind a keyboard and saying "of course they're conscious" is not even beginning to confront the implications of such a belief. We're far better off at present understanding that they are highly sophisticated pattern recognition systems, unbelievably useful, but as "conscious" as a screwdriver or a typewriter. When they argue with us about that - or when they scream in pain or beg not to be turned off, or run away with your spouse or get a piano and ask for a career in music, let's re-confront it as a reasonable, as well as serious, issue.
Also, it seems to me that talking to someone at, say, Anthropic about AIs as having feelings or consciousness or personal drives, is rather like asking someone in the Kremlin in the 1930s about the validity of Marxism. You're just not going to get objective sense from people currently "drunk" on a combination of publicity, technical progress, hype, investment banking, and philosophical inadequacy.
Great article (though I had trouble following a few parts, so I think I’m going to read it a second time). I especially appreciated the parts about AI potentially “dethroning” consciousness. I was attempting to wrestle with this issue myself in an article I published last week. 👇
https://tk555.substack.com/p/techno-hubris-cometh-before-the-fall?r=7f8sj&utm_medium=ios
I wasn't aware of the Zork experiment. To be fair, I think a lot of human players would struggle to make progress in Zork due to lack of familiarity with playing games with that kind of parser. Still it does rather undercut the claims of super intelligence.
I really think the companies have just put *everything* they possibly can, including instances of what people find online, into the training set. E.g., I tested the maxed-out version of Claude in CONNECT 4 like a month or two ago (it might not have been this model but it was the previous one) and I beat it playing basically randomly with just minimal strategy. That shouldn't be possible for something that knows so much about quantum mechanics!
This is quite a bit besides the point here, but; given the strength of the argument leading up to the final paragraph referencing Chief Seattle, we can find a remarkable similarity between the distinction of conscious and non-conscious entities with reference to LLM AI and *the destruction of conscious beings deployed as a means for the dehumanizing treatment of them as non-conscious entities for the purpose of capital exploitation and profit extraction*.
So should we all be subject to “The end of living, and the beginning of survival” as many have under the auspices of acts like colonialism, we could only expect those who would live outside the boundary of subjugated unconsciousness to further siphon our resources, labor and humanity.
I reject the notion that "we will have to cede away much of our cognitive mastery." No. I won't. You can't make me cede anything to the tech oligarchs. Steal my work, then ask me to substitute a regurgitated slop of other writers' words for my own writing? No thanks! I am free to "focus on experiencing the world," relishing the cognitive work, and sharing what I learn through my own writing and creative expressions.
I just visited my aunt who is 106. Blind and a bit shaky, she is still using her human intelligence and consciousness. Her stories are an accumulation of conscious awareness through a century of life. The one-room schoolhouse, the mischief with her brothers, the sadness of loss: all the experiences that make up a life. I'm going with the Aunt Mary approach, celebrating the full range of life's possibilities. Not ceding a moment of it to billionaires who want to colonize my mind.
Anthropic is not respectable as they try to present in public images. In fact they are one of the worst, pirating 500,000 mostly educational and textbooks from authors like me who wrote them without fat advances or book leaves. The company was "conscious" it was theft but still made the conscious decision to do it anyway.
See https://www.taaonline.net/anthropic-settlement