One lesson from math is that conjectures can be proven to be false. You're not a loser if you show a conjecture is false.
Similarly, "consciousness can have a scientific explanation" is false if science is taken to be the study of what is objectively observable. Because consciousness cannot be directly observed by third parties.
I believe you brought this up in your book, but I don't remember if you found a way around it.
Yeah, that might be one outcome: but right now we’d be in the Russellian “the set of all sets that don’t contain themselves” phase of intuitions (which you’re right, I have written about). It’s a long ways to an actual proof. And that proof would be almost as big as solving the problem itself - it would likely have bearings on a ton of other things. In fact, we’d understand the issue so much better that I would almost count that in the “solved” camp!
> "consciousness can have a scientific explanation" is false if science is taken to be the study of what is objectively observable.
I am saying that this is easy to say (and I've said similar things myself), but it's a totally different level to get something that convinces people widely and/or is formal and extended, i.e., something accepted by the scientific community at large or a direct proof. That itself would be massive progress. It would likely look more like Gödel's proof in terms of the level of impact and being very different and much more precise than how people (including myself) currently talk about these things.
P-zombies are a conceivability argument, and conceivability arguments are (a) interesting and valid to talk about, but (b) if you are convinced by them *to the degree that you wouldn't fund scientific research into consciousness* this is an epistemic mistake. E.g., prior to the development of quantum physics, it would have been extremely easy to give conceivability arguments against many aspects of the theory. It would likely have been inconceivable that something can be *both* a particle and a wave. How is that possible? It's paradoxical. And yet, it works. What's most likely is that we get a theory of consciousness where brains can be both objective and subjective at the same time, and this seems paradoxical. And yet, it works.
Fine, substitute LLM for p-zombie. What is the test for consciousness of an LLM?
There is none, and cannot be one, even in principle.
You can have THEORIES, but it is impossible to TEST the theories experimentally. So if two theories disagree about whether a particular LLM is conscious or not, there is no possible experiment to deetermine which theory is right.
The situation of quantum physics is not comparable. It was not at all "inconceivable" that something can be both a wave and a particle, this was conceived as soon as the experimental evidence required it. The issue was then to provide math that allowed for that, which was eventually done, and then tested against many other experiments. This back-and-forth between theory and experiment is vital to science, was crucial for quantum physics, and is impossible for consciousness.
I got around Godel's incompleteness theorems and Munchhausens Trilemma, whilst solving why reality persists and multiple paradoxes like infinite regression and entropy, and more Turned out there is an answer, I just approached it from a different angle.
That is something of a mystery to me, but I know what you're talking about. Why do I need to show no one has observed consciousness? Isn't that self-evident?
I bet this is related to Western civilization failing to come up with the number 0. Like, a fundamental difficulty with grasping an abscence, a desire to fill in all the blanks.
Your comment that "the ideas are fun to discuss on podcasts, and no opinion can really be proven wrong" captures why I (a layman) first became interested in consciousness as an area of study, and, after a while, burned out. It can feel like anything goes -- and that even with experts, popular discussions lack rigor and bend towards guru dynamics...
It’s both understandable, given the difficulty of the topic, but it’s also pernicious. There are people who actively trying hard but they are often masked by the “talkers.” Social media has made this surprisingly worse: there are a number of people who are considered high-profile in consciousness research but if you check… they don’t actually do any original research!
But in a weird way, I think this should update people in a positive direction. The more constraints we can point to as historical / social / structural contingencies the more likely the problem is mostly those. Like where is the “great filter” for the field? Is it really one big thing we’re circling around, or is it a ton of little filters stacked together, stopping progress?
History seems replete with examples of people engaging with the consciousness question without doing research. This actually seems to be in line with the point you are making. I, for one, have been frustrated by the lack of rigor on this topic for decades. Science seems to have been avoiding this topic for reasons other than it is hard.
I think a true understanding would involve something similar to Gödel's proofs, but as applied to science itself. Showing this is really the case, and exactly why, and working it out in such a way as to convince many others, would be a monumental intellectual achievement and tell us a lot about other areas of science. And given that we have nothing like that, just some half-baked arguments (I'd include some of mine in this description, as I think I'm the originator of one or two now-more-popular arguments for mysterianism), it means there is still a ton of work to do, so the standard version of mysterianism or the existing versions would still be defeatist in my view.
I truly appreciate your fervor. I studied philosophy of mind while minoring in philosophy, went on to mathematical neuroscience, and now work more broadly in computational neuroscience. While I mostly gave up on a science of C as a lost cause long ago, I continue to keep up with the field out of interest. Nearly nothing of substance has changed.
On the one hand, I continue to keep an open mind. On the other, it seems that we are so far from being capable of examining phenomenal consciousness—PC—that, short of a visit by aliens or (possibly unconscious and insect-like, in your words) post-singularity ASI, it will and should remain a parlor discussion.
Why? Because unless PC can cause observable and repeatable changes in a system in a way that cannot be predicted without assuming PC, we will never have reason to add it to our physics. If Dennett is right, for example, and PC is really just AC, access consciousness, from the inside, then there’s nothing to explain or add. If he’s wrong but it’s non-causal or non-measurable (like is the case currently), then it’s unnecessary to add to explain the objective world around us. In either scenario, we’re back to the realm fundamental PC still firmly belongs: philosophy.
We can and should further study the neural correlates of C, both AC and PC. We can and will make systems that undeniably have AC. But without measurable causality, the science of PC will necessarily remain stuck in studies of neural correlates and arguments from analogy.
I think part of the issue is that as humans, PC is so fundamental to our existence that we greatly romanticize it and therefore assume that there has to be more to it than as merely a side effect of the complex modelling our brains are engaged in. Coming to this position has helped me downgrade consciousness as a hard problem. A worthy successor in my view is the Hard Problem of Happiness.
I have a friend with a similar view. I certainly find it humbling that so many intelligent, informed people come to such different, and usually very firmly held, conclusions. As a result, while I have my own thoughts, I do my best to be ready to change my mind and refine my reasoning as I read and think over new material.
The Hard Problem of Happiness: Love it. In fact I’d almost want to simply dub it The Harder Problem.
I’m sure your experience of Pollan’s book was largely colored by the fact that you swim in these waters constantly and were reminded of the lack of progress in the field.
As a layperson who is simply interested in discussions on the topic, I enjoyed it. It helped me think about consciousness in new ways and I would put it in a similar vein as the audio documentary “Lights On.” Meant as a popular exploration of the state of consciousness research, not as an explanation. Perhaps it will help increase funding because the public will be more aware of the issue.
I would think Pollan would come to a conclusion similar to your own, but the valence would be more positive/contemplative than your understandable frustration.
I hope to see what consciousness researchers can come up with if funding increases significantly.
Curious why you chose to call it Bicameral Labs. Do you think that the answer will end up being Jaynesian? Do you think bicameral breakdown has anything to do with the end of the Sapient Paradox?
I’m having the same crisis reading Eric Ries’ “Incorruptible” (recommended to me by so many people.)
It’s…. not bad. It’s good for people to know about these case studies. But it’s overwhelmingly pop nonfiction with little substance and an overtendency to boil good companies down to “has a good mission” “is trustworthy” and so on.
I’d imagine it’s quite similar for you reading Pollan on consciousness 😆. Let’s make more progress than this….
thank you for this. 24yF 3rd yr med student here, who was raised in awe and fascination of my mentors in consciousness research & cognitive science, who's been feeling so disenchanted by the academic discipline and its passiveness to what's happening.
the space is so beautiful for its openness and flexibility of mind, but it also means there's a certain hands-up in the face of the unknowns.
Eric- I've loved reading your work, and have learned so much. Thank you. However, today you completely lost me. This is your definition of consciousness: "Erik Hoel: “Consciousness is what is like to be you. It is the set of experiences, emotions, sensations, and thoughts that occupy your day, at the center of which is always you, the experiencer. You lose consciousness when you go under anesthesia and when in a dreamless sleep, and you gain consciousness when you wake up in the morning or come up from anesthesia.”
Eric, you will never know what it is like to be me, and I will never know what it's like to be you. Therefore, it seems to me that your definition of consciousness is something I can agree with, but it is functionally useless.
Thanks! I think it's actually totally possible to have a science of consciousness if we can't personally know what it's like to be another person (or a bat, or an AI, etc.). I think even without that a science of consciousness could still (a) answer why this what-it-is-likeness exists in this system, as in what necessitates it, what the sufficient conditions are, etc. and (b) use that to understand how the system works functionally, and (c) allow us to read out objective descriptions of subjective experiences (e.g., seeing red) in a lawful way, rather than a correlative way (which we can now). Just to see a real concrete example, (c) would be extremely useful for locked-in patients, who can't communicate that they are conscious.
Wouldn’t it be great if there could be a global (or even local) Moon Shot for consciousness? Yes, as with all things, we’ve swung from too little decades ago to too much today (over 350 theories!). And way too few have been tested.
It’s very possible that the p(solve consciousness) basically just goes up and up with funding, such that p(solve consciousness) for like, 100 million dollars, might be shocking high and it’s mostly just various contingencies that have kept this from happening.
There is absolutely nothing strange about an expert hating a well-written popular book about their field. I trained as a mathematician and physicist and worked as a computer scientist and all the popular books about Chaos Theory and the meaning of quantum mechanics and how AI will change the world are unreadable.
Erik, thanks for letting me in. I've been away, unfortunately cancelled from "paid." I posed the following question below to generative Ai, because it seems that as a social science humanist, the entire enterprise of the science of "consciousness" lacks its companion, humanity. As an aside, as I am a big fan, I wonder why you don't include eco-poets like Gary Snyder, in part to complement the inclusion of philosophers. Gary's view is eons long, dating to the Paleolithic. In "The Practice of the Wild," and other works spanning nine + decades, Snyder refers to the 'Natural Mind." Highly worth considering, even for objectively verifying scientists (David Bohm would agree in my judgment).
Thus, my question: “What does LLM mean in the science of consciousness? Why do researchers like Erik Hoel rely on this term so extensively, seeming to the point--hate to say this,--obsessively? Is the term that significant to Hoel and others that without it, no definition of consciousness is possible; therefore no objective measurement of this science can exist: Thousands of (failed) studies without the possibility of proof? Are Hoel and others in the field right? It seems that the whole academic-like enterprise leaves out the role of humanity, even though Hoel includes the significance of being a major outcome to strive for.”
AI Response:
Is he (YOU) right, or is humanity left out?
Critics and peers often fall into two camps regarding Hoel's approach:
* The "Proof" Problem: Many agree that without objective measurements, the study remains a "soft" science. However, some argue his "disproof" of LLM consciousness relies on assuming his own definitions of "learning" and "falsifiability" are the only valid ones.
* The Human Element: You are right that this can feel like an "academic enterprise" that reduces humanity to math. Hoel himself acknowledges this tension; he has noted that science has historically "bracketed" the soul to make progress, and his work is an attempt to bring that "soul" back into the light of rigorous calculation without losing its essence. [1, 2, 3, 4, 5]
Ultimately, for Hoel, the LLM is a mirror: if we can't prove why a computer isn't conscious, we can't truly claim to understand why a human is.
The above is further proof is you've long ago made It to "the show." Nice work!!--Luis
As you may guess, Zero citations in the overview of your work, Erik. Your ilk has no room for the ambiguous words, soul.' The term "being" as in humans is, apparently. That's a good thing to me. (Also it's beyond my comprehension how you can do both science and fiction. Amazing).🥲
One lesson from math is that conjectures can be proven to be false. You're not a loser if you show a conjecture is false.
Similarly, "consciousness can have a scientific explanation" is false if science is taken to be the study of what is objectively observable. Because consciousness cannot be directly observed by third parties.
I believe you brought this up in your book, but I don't remember if you found a way around it.
Yeah, that might be one outcome: but right now we’d be in the Russellian “the set of all sets that don’t contain themselves” phase of intuitions (which you’re right, I have written about). It’s a long ways to an actual proof. And that proof would be almost as big as solving the problem itself - it would likely have bearings on a ton of other things. In fact, we’d understand the issue so much better that I would almost count that in the “solved” camp!
Not sure I follow. An actual proof of what?
> "consciousness can have a scientific explanation" is false if science is taken to be the study of what is objectively observable.
I am saying that this is easy to say (and I've said similar things myself), but it's a totally different level to get something that convinces people widely and/or is formal and extended, i.e., something accepted by the scientific community at large or a direct proof. That itself would be massive progress. It would likely look more like Gödel's proof in terms of the level of impact and being very different and much more precise than how people (including myself) currently talk about these things.
The scientific community already knows that consciousness is untestable, that's why you don't get any money!
There is no conceivable experimental test that can distinguish a p-zombie from a truly conscious entity.
Disagree? The burden of proof is on you.
P-zombies are a conceivability argument, and conceivability arguments are (a) interesting and valid to talk about, but (b) if you are convinced by them *to the degree that you wouldn't fund scientific research into consciousness* this is an epistemic mistake. E.g., prior to the development of quantum physics, it would have been extremely easy to give conceivability arguments against many aspects of the theory. It would likely have been inconceivable that something can be *both* a particle and a wave. How is that possible? It's paradoxical. And yet, it works. What's most likely is that we get a theory of consciousness where brains can be both objective and subjective at the same time, and this seems paradoxical. And yet, it works.
Fine, substitute LLM for p-zombie. What is the test for consciousness of an LLM?
There is none, and cannot be one, even in principle.
You can have THEORIES, but it is impossible to TEST the theories experimentally. So if two theories disagree about whether a particular LLM is conscious or not, there is no possible experiment to deetermine which theory is right.
The situation of quantum physics is not comparable. It was not at all "inconceivable" that something can be both a wave and a particle, this was conceived as soon as the experimental evidence required it. The issue was then to provide math that allowed for that, which was eventually done, and then tested against many other experiments. This back-and-forth between theory and experiment is vital to science, was crucial for quantum physics, and is impossible for consciousness.
I got around Godel's incompleteness theorems and Munchhausens Trilemma, whilst solving why reality persists and multiple paradoxes like infinite regression and entropy, and more Turned out there is an answer, I just approached it from a different angle.
That is something of a mystery to me, but I know what you're talking about. Why do I need to show no one has observed consciousness? Isn't that self-evident?
I bet this is related to Western civilization failing to come up with the number 0. Like, a fundamental difficulty with grasping an abscence, a desire to fill in all the blanks.
Your comment that "the ideas are fun to discuss on podcasts, and no opinion can really be proven wrong" captures why I (a layman) first became interested in consciousness as an area of study, and, after a while, burned out. It can feel like anything goes -- and that even with experts, popular discussions lack rigor and bend towards guru dynamics...
It’s both understandable, given the difficulty of the topic, but it’s also pernicious. There are people who actively trying hard but they are often masked by the “talkers.” Social media has made this surprisingly worse: there are a number of people who are considered high-profile in consciousness research but if you check… they don’t actually do any original research!
But in a weird way, I think this should update people in a positive direction. The more constraints we can point to as historical / social / structural contingencies the more likely the problem is mostly those. Like where is the “great filter” for the field? Is it really one big thing we’re circling around, or is it a ton of little filters stacked together, stopping progress?
History seems replete with examples of people engaging with the consciousness question without doing research. This actually seems to be in line with the point you are making. I, for one, have been frustrated by the lack of rigor on this topic for decades. Science seems to have been avoiding this topic for reasons other than it is hard.
“Consciousness is not here for our personal therapy. It’s not tied to our life journeys.”
Well, maybe it is.
Eric, I hope your fate is better than Shackleton's!
Wasn’t shackletons fate positive? He might not have achieved his goals he initially set out but the voyage was incredible nevertheless.
He died trying to achieve his goal. Maybe not a bad way to die.
Mysterianism might be defeatist but it might also be right. It certainly remains viable.
I think a true understanding would involve something similar to Gödel's proofs, but as applied to science itself. Showing this is really the case, and exactly why, and working it out in such a way as to convince many others, would be a monumental intellectual achievement and tell us a lot about other areas of science. And given that we have nothing like that, just some half-baked arguments (I'd include some of mine in this description, as I think I'm the originator of one or two now-more-popular arguments for mysterianism), it means there is still a ton of work to do, so the standard version of mysterianism or the existing versions would still be defeatist in my view.
If someone wants to become involved in the solution as you describe it, most likely via Bicameral Labs, what's the best way to do so?
Send an email to erik@bicameral-labs.org!
I truly appreciate your fervor. I studied philosophy of mind while minoring in philosophy, went on to mathematical neuroscience, and now work more broadly in computational neuroscience. While I mostly gave up on a science of C as a lost cause long ago, I continue to keep up with the field out of interest. Nearly nothing of substance has changed.
On the one hand, I continue to keep an open mind. On the other, it seems that we are so far from being capable of examining phenomenal consciousness—PC—that, short of a visit by aliens or (possibly unconscious and insect-like, in your words) post-singularity ASI, it will and should remain a parlor discussion.
Why? Because unless PC can cause observable and repeatable changes in a system in a way that cannot be predicted without assuming PC, we will never have reason to add it to our physics. If Dennett is right, for example, and PC is really just AC, access consciousness, from the inside, then there’s nothing to explain or add. If he’s wrong but it’s non-causal or non-measurable (like is the case currently), then it’s unnecessary to add to explain the objective world around us. In either scenario, we’re back to the realm fundamental PC still firmly belongs: philosophy.
We can and should further study the neural correlates of C, both AC and PC. We can and will make systems that undeniably have AC. But without measurable causality, the science of PC will necessarily remain stuck in studies of neural correlates and arguments from analogy.
I think part of the issue is that as humans, PC is so fundamental to our existence that we greatly romanticize it and therefore assume that there has to be more to it than as merely a side effect of the complex modelling our brains are engaged in. Coming to this position has helped me downgrade consciousness as a hard problem. A worthy successor in my view is the Hard Problem of Happiness.
I have a friend with a similar view. I certainly find it humbling that so many intelligent, informed people come to such different, and usually very firmly held, conclusions. As a result, while I have my own thoughts, I do my best to be ready to change my mind and refine my reasoning as I read and think over new material.
The Hard Problem of Happiness: Love it. In fact I’d almost want to simply dub it The Harder Problem.
I’m sure your experience of Pollan’s book was largely colored by the fact that you swim in these waters constantly and were reminded of the lack of progress in the field.
As a layperson who is simply interested in discussions on the topic, I enjoyed it. It helped me think about consciousness in new ways and I would put it in a similar vein as the audio documentary “Lights On.” Meant as a popular exploration of the state of consciousness research, not as an explanation. Perhaps it will help increase funding because the public will be more aware of the issue.
I would think Pollan would come to a conclusion similar to your own, but the valence would be more positive/contemplative than your understandable frustration.
I hope to see what consciousness researchers can come up with if funding increases significantly.
Thanks for the article.
Curious why you chose to call it Bicameral Labs. Do you think that the answer will end up being Jaynesian? Do you think bicameral breakdown has anything to do with the end of the Sapient Paradox?
I’m having the same crisis reading Eric Ries’ “Incorruptible” (recommended to me by so many people.)
It’s…. not bad. It’s good for people to know about these case studies. But it’s overwhelmingly pop nonfiction with little substance and an overtendency to boil good companies down to “has a good mission” “is trustworthy” and so on.
I’d imagine it’s quite similar for you reading Pollan on consciousness 😆. Let’s make more progress than this….
thank you for this. 24yF 3rd yr med student here, who was raised in awe and fascination of my mentors in consciousness research & cognitive science, who's been feeling so disenchanted by the academic discipline and its passiveness to what's happening.
the space is so beautiful for its openness and flexibility of mind, but it also means there's a certain hands-up in the face of the unknowns.
Eric- I've loved reading your work, and have learned so much. Thank you. However, today you completely lost me. This is your definition of consciousness: "Erik Hoel: “Consciousness is what is like to be you. It is the set of experiences, emotions, sensations, and thoughts that occupy your day, at the center of which is always you, the experiencer. You lose consciousness when you go under anesthesia and when in a dreamless sleep, and you gain consciousness when you wake up in the morning or come up from anesthesia.”
Eric, you will never know what it is like to be me, and I will never know what it's like to be you. Therefore, it seems to me that your definition of consciousness is something I can agree with, but it is functionally useless.
Thanks! I think it's actually totally possible to have a science of consciousness if we can't personally know what it's like to be another person (or a bat, or an AI, etc.). I think even without that a science of consciousness could still (a) answer why this what-it-is-likeness exists in this system, as in what necessitates it, what the sufficient conditions are, etc. and (b) use that to understand how the system works functionally, and (c) allow us to read out objective descriptions of subjective experiences (e.g., seeing red) in a lawful way, rather than a correlative way (which we can now). Just to see a real concrete example, (c) would be extremely useful for locked-in patients, who can't communicate that they are conscious.
Wouldn’t it be great if there could be a global (or even local) Moon Shot for consciousness? Yes, as with all things, we’ve swung from too little decades ago to too much today (over 350 theories!). And way too few have been tested.
It’s very possible that the p(solve consciousness) basically just goes up and up with funding, such that p(solve consciousness) for like, 100 million dollars, might be shocking high and it’s mostly just various contingencies that have kept this from happening.
There is absolutely nothing strange about an expert hating a well-written popular book about their field. I trained as a mathematician and physicist and worked as a computer scientist and all the popular books about Chaos Theory and the meaning of quantum mechanics and how AI will change the world are unreadable.
Pollan is on Sam Harris's podcast - out today - fwiw. Haven't listened yet.
Erik, thanks for letting me in. I've been away, unfortunately cancelled from "paid." I posed the following question below to generative Ai, because it seems that as a social science humanist, the entire enterprise of the science of "consciousness" lacks its companion, humanity. As an aside, as I am a big fan, I wonder why you don't include eco-poets like Gary Snyder, in part to complement the inclusion of philosophers. Gary's view is eons long, dating to the Paleolithic. In "The Practice of the Wild," and other works spanning nine + decades, Snyder refers to the 'Natural Mind." Highly worth considering, even for objectively verifying scientists (David Bohm would agree in my judgment).
Thus, my question: “What does LLM mean in the science of consciousness? Why do researchers like Erik Hoel rely on this term so extensively, seeming to the point--hate to say this,--obsessively? Is the term that significant to Hoel and others that without it, no definition of consciousness is possible; therefore no objective measurement of this science can exist: Thousands of (failed) studies without the possibility of proof? Are Hoel and others in the field right? It seems that the whole academic-like enterprise leaves out the role of humanity, even though Hoel includes the significance of being a major outcome to strive for.”
AI Response:
Is he (YOU) right, or is humanity left out?
Critics and peers often fall into two camps regarding Hoel's approach:
* The "Proof" Problem: Many agree that without objective measurements, the study remains a "soft" science. However, some argue his "disproof" of LLM consciousness relies on assuming his own definitions of "learning" and "falsifiability" are the only valid ones.
* The Human Element: You are right that this can feel like an "academic enterprise" that reduces humanity to math. Hoel himself acknowledges this tension; he has noted that science has historically "bracketed" the soul to make progress, and his work is an attempt to bring that "soul" back into the light of rigorous calculation without losing its essence. [1, 2, 3, 4, 5]
Ultimately, for Hoel, the LLM is a mirror: if we can't prove why a computer isn't conscious, we can't truly claim to understand why a human is.
The above is further proof is you've long ago made It to "the show." Nice work!!--Luis
Just focusing on one thing: Where does it think I say anything about souls? Like what are those citations?
As you may guess, Zero citations in the overview of your work, Erik. Your ilk has no room for the ambiguous words, soul.' The term "being" as in humans is, apparently. That's a good thing to me. (Also it's beyond my comprehension how you can do both science and fiction. Amazing).🥲