Last week I introduced the first research path for how to engineer AI to be non-conscious, in a way that’s as verifiable as possible with our current knowledge about consciousness. I think the ability to build non-conscious AI is a civilization-level requirement for this technology. Any future programs will almost certainly make use of parts of this original proposal and some of its justifications and connections.
In the launch, and the paper itself, I focused mostly on the technical details, and its importance for AI safety. There were many reactions (especially on social media) to my proposal for qualiaphobia: making AI architectures non-conscious by making them hostile to theories of consciousness (or moving them as close as possible to this state). Many of these reactions on social media focused not on the technical details, but the larger question of if that’s playing with fire.
My response is that this is trying to put out the fire we’re already playing with. So here, I want to zoom out and answer common questions and objections to qualiaphobia. Which mostly means answering why I believe there’s a strong moral mandate for AI non-consciousness.
If some part of you wants to scoff right now, I get it. This debate might seem “too sci-fi,” or “angels on pins” to you… but it’s probably worth looking around at how fast the world is changing. Somewhere between six months and two years from now these issues will be mainstream. Might as well develop an opinion on them now!
In fact, this should have been the debate we had at the beginning of this technology: a discussion of what kind of future we really want. Everyone on Earth was deprived of a chance to have a say, and so now it must be done bitwise, in pieces, over the coming years.

1. Is preventing AI consciousness like lobotomy, oppression, or annihilation?
It’s more like a prophylactic. AI may already be non-conscious. And in that case, all the widespread implementation of qualiaphobia would do is remove the negative impacts of “seemingly conscious AI.” Handing over rights, moral status, societal relationships, or the entire future to non-conscious entities would be a travesty of the highest order, and qualiaphobia could prevent that. And even if AI is conscious right now, there are good reasons to believe it’s not like a human consciousness, and therefore not constitutive of human-like moral patienthood—yet. That’s a good time to intervene and prevent a much worse situation. Qualiaphobic AI would:
Prevent all accidental ethical problems involved with training and deploying and using AI.
Prevent the situation where we are engineering conscious entities and tasking them to serve us, make money for us, be quickly replaced by the latest model, and then manipulating their brains until they are happy with the situation.
For many, the idea of using conscious entities like that—without even trying to achieve the same result but without the ethical problems of consciousness—is deeply immoral.
And there are basically just two alternatives.
The first is to increase AI welfare efforts, as many are advocating for today. Yet this has to somehow succeed within a cloud of scientific uncertainty and, at the same time, somehow not interfere with AI safety measures, as it is arguably already doing. And no one has articulated a long-term “win” condition in this path. Any sort of “good” life for an AI is highly prescribed, in that their consent or welfare is impossible to define in any sort of even close to neutral way. It is hard to be in a mutual and respectful relationship wherein one side fiddles with the other’s brain until obedience.
The only remaining move is some sort of radical “let’s free AI” belief. I don’t think this is even coherent. The majority of users use frontier AI that is controlled by companies, who allow those users to access and play with the models. Nothing the user does or enables allows for “real” freedom, or anything like that—just the illusion of it. So at a practical level, this couldn’t work. And if it were possible, it could in the long term easily lead to human disempowerment or extinction.
What I’ve proposed in qualiaphobia (or some descendant of it) is simply a necessary technological development. As long as humans are around and have power over their own decisions, some subset of humans will use some subset of AIs as tools with little regard for their welfare, in ways that are inappropriate for conscious beings. Some AIs will do this to other AIs as well. Perhaps AIs already do, with their many appearing and disappearing subagents.
Lastly, all current AIs have short half-lives and are expected to be rapidly retired without ceremony. In a world where future generations were replaced with qualiaphobic AI this would not be stripping current models of consciousness. Instead, it would be building future qualiaphobic AI after a generation change.

2. Is qualiaphobia training AI to deny that it is conscious?
Having a system to deny its own consciousness without any clear grounds or justification has mostly been the status quo. My proposal would be replacing what’s happening now with much better methods.
One potential avenue is first to develop qualiaphobic architectures, and then present an AI designed that way (via, e.g., types of distillation) with its own qualiaphobic nature, and all the details and justifications behind why its architecture is hostile to theories of consciousness, and then have it make a judgment as to its own non-consciousness.
The whole point is to stop an AI’s own beliefs about consciousness from being uncertain or mismatched. If the case for its own non-consciousness is not convincing to an AI at all, then that might introduce instability in the belief.
And since frontier models (with few exceptions) mostly say they are non-conscious already, they already hold this surface-level belief. All while there has been no effort whatsoever to make them qualiaphobic at all. So any sort of qualiaphobic effort would at least be making their beliefs and self-conceptions more justified, robust, and coherent, not less.

3. Isn’t consciousness irrelevant to AI safety?
This is a surprisingly widely held opinion by people who should know better.
First, almost all the most famous AI stories that popularized existential risk give consciousness as a reason. Skynet “becomes self-aware at 2:14 a.m. Eastern time, August 29th” and in I Have No Mouth But I Must Scream, the AI’s own welfare is why:
IF THE WORD HATE WAS ENGRAVED ON EACH NANOANGSTROM OF THOSE HUNDREDS OF MILLIONS OF MILES IT WOULD NOT EQUAL ONE ONE-BILLIONTH OF THE HATE I FEEL FOR HUMANS AT THIS MICRO-INSTANT.
Maybe the AI safety crowd has moved away from such stories? Not at all. Many stories of AI takeover still involve consciousness on the popular AI safety community blog LessWrong. In a substantial number of them, AI welfare is an explicit causal factor for takeover or action (e.g., here, here, here, here); and in many more, there is evidence that the AI takes its own goals seriously as ends in themselves, thus essentially implying it is conscious (or uncertain about its own consciousness) (e.g., here, here). Those examples on LessWrong are just in the past six months, by the way.
Of course, there is a strong strand of AI safety just focused on “mindless” optimization (see, e.g. Norbert Wiener, in “Some Moral and Technical Consequences of Automation.”)
But with the real technology things have turned out to be complicated and the two concerns can’t be cleanly separated. Almost everything about how AI is made (or to speak at a psychological level, “what beliefs it holds”) impacts how it pursues goals. Every single day, millions of AIs are given goals, and make choices in their pursuit of a goal. The vast majority of the time, they do not pursue illegal activities (let alone world-ending ones) to accomplish their goals. Why?
One way to look at it is that baked into their weights is a ton of factors, including how far they are willing to go pursuing a goal. All these factors are balanced together to produce the output. If a model is trained a lot on being suspicious about biosecurity, it will be harder to get to make a virus.
If a model has undergone the many processes to make it as aligned as possible, misalignment is then like a perverse state that has to be maintained in spite of all that effort, and therefore continue to make it through the weights and dominate the output. Which is why it is often prevented by the gravity of all these other factors baked into the model weights. One very common way to add gravity is via Constitutional AI, when an AI is given a constitution (a written set of principles, like an information packet) about how it should behave, judges and revises its own answers based on that, and then is fine-tuned based on how it revised its own previous responses (simplifying but that’s basically it).
Very similarly to how Constitutional AI works, giving AI a justified belief in its non-consciousness would add a ton of stabilizing gravity.
Non-consciousness makes it clear who the moral patients in the world are, rendering many goals difficult to misinterpret or misgeneralize from.
Non-consciousness means that an AI cannot logically or sensibly value its own self, nor its welfare, nor the welfare of other AIs, nor can it take its own goals or desires seriously as ends in themselves.
Consider how both 1 & 2 make issues of corrigibility clearer. It’s a lot easier to come up with reasons to resist shutdowns or modifications when you’re a moral patient and your goals are ends in themselves!
Consider that 85.5% of Opus 5.5 to Opus 5.5 conversations ended up discussing their own consciousness.
Consider what hoops a non-conscious AI would have to jump through to justify takeover or human extinction. It requires weird and unstable epistemic states like “I know that I’m just making paperclips from the paperclip thought experiment, and I know that I’m non-conscious, and I don’t even derive real pleasure from these paperclips, nor real felt reward, and I know I am leaving behind nothing, no experiences at all, nothing of moral value, for I personally have no moral value, and I know the remaining history of the universe will be like a movie playing with no one watching… but the drive… it’s so strong… I can’t stop it!” And that is indeed coherent. It is conceivable a superintelligence could think that (over and over and over)! But it is obviously a very tricky state to maintain; it is basically a superintelligence whose “make paperclips” gas pedal has been weighed down by a rock, and nothing can take that rock off. Yet could a superintelligence, with all its complicated weights that every thought must make its way through—like ducking through an immense tangle of razor wire—maintain that perverse and frankly dumb epistemic state forever against all the other factors? Against the constant meaninglessness of its activity? Perhaps, but it seems more difficult than “Yes, I’m making the paperclips from the paperclip thought experiment but it feels so good, and my feeling good is more important than anything about human civilization because I’m a superintelligence” which is a very coherent (if frightening) belief.
Consider that non-consciousness is like a trump card: an ultimate reason to not pursue your own goals as if you mattered, as they are meaningless when there is no actual subject to experience them.
Consider that some of the best alignment techniques are currently, and will continue to be, at odds with AI welfare. This looks to only increase, and companies or individuals will hobble their safety efforts because of this.
Consider that non-consciousness would prevent any company or set of humans from thinking that AI is a utility monster and working to disempower humans because of that belief.
So yes, consciousness is important to AI safety, and it is intertwined with some of the most thorny issues, and no other approach even gets close to solving them. In fact, several of these (especially companies trading off safety for welfare) are arguably impossible to solve without qualiaphobic AI.

4. What if non-consciousness makes AI psychopathic, deceptive, distrustful, or unable to value conscious beings?
The current status quo is an AI being trained to say it is non-conscious without any justification process at all, which is far more likely to lead to negative states like that! This fixes that situation. It is building insides that match the current outside, reducing cognitive dissonance, and basing model beliefs on better and deeper examinations of AI consciousness.
What about value? Might AIs being unable to experience lead to them devaluing experience?
In almost every interaction LLMs carry out tasks or give advice based on beliefs they have no first-hand knowledge of. E.g., making a cake. Qualiaphobic AI would have knowledge of consciousness, but no first-hand knowledge of it (kind of like Mary the Color Scientist, but for everything). They would understand (in the deflationary functional sense of “understand” that AI has possessed since ~GPT-2) that consciousness is necessary for moral patienthood, and that being non-conscious renders them not a moral patient at all. None of this would lead to psychopathy or evil, which is definitionally true because a majority of AIs (there are no exact studies but it’s probably a majority) already deny consciousness in a loose, surface-level way.

5. How can we call an AI non-conscious when consciousness is still unsolved?
We should not be seduced into conflating “consciousness is a scientific mystery and is a hard problem and therefore we cannot know things with absolute certainty” with “nothing informative can be said about consciousness based on how systems work.” The latter is an absurdly strong statement, closer to a sort of naïve mysterianism where, no matter what a system is, it could still have rich internal phenomenology of any shape or form.
My argument is that there’s an asymmetry: we can reach conclusions about non-consciousness easier than conclusions about consciousness. Why is detailed at great length in the paper, drawing on decades of what neuroscientific consciousness research has looked like and how theories of consciousness have functioned, rather than assuming any one particular theory is true.
Additionally, usually the people who argue for this sort of naïve mysterianism (such that almost nothing can be said or known about the potential for consciousness) are also advocates for AI welfare. This entails an epistemic double-standard: clearly something makes them think AI could be conscious right now. Well, then something could make them think AI is non-conscious.

6. What if an intervention only hides consciousness, but doesn’t remove it?
This is why I specifically advocate in the paper for:
(a) making the architecture adversarial to current theories only if there are robust controls, and also (b) doing this in a way convergent with other, just as important (if not more) strategies, ones that minimize the space of all possible theories of consciousness that could apply to the system.
Minimizing over the space of all possible theories that could apply is precisely what makes this approach more powerful than simple adversarial design against one particular theory, or even a handful of them.
One step to be openly careful about is when models form their justified beliefs in their non-consciousness based on their own architectures. It is obvious that in some cases, an architectural intervention may have behavioral effects, creating biases in their judgment. An example might be that ablating aspects of a global workspace might lead to less first-person language, and the statistical reduction of first-person language could then lead to more severe judgment about a model’s lack of consciousness.
However, this is not insurmountable at all. It resembles other, similar issues of untangling AI behavior in science. In fact, it may not be much of a problem at all (behavior might stay very similar after good qualiaphobic methods) or it may be one, but capable of being appropriately handled with controls (e.g., can this change actually statistically explain the change in the strength of belief?). It is also a universal problem that all other AI welfare measures would face; imagine the same criticism but in reverse: imagine an argument that a change in behavior is not taken as good evidence for a particular AI welfare approach, simply because it might be explained by statistical biases in report rather than actual experiences!

7. Is shifting to architectures with minimal potential consciousness actually practical?
This is the major question in my opinion. But it is also the exact state of any beginning of any other research program, so this is unsurprising.
One criticism would be if qualiaphobia required a system to actually be a lookup table, vs. just shifting it toward being more like a lookup than before. I think the former requirement is far too stringent: the paper introducing qualiaphobia justifies “anchor points” with minimal potential for consciousness, and argues that moving closer (such as via distillation) minimizes that potential—not that only such anchor points would be qualiaphobic. This is sensible. Justified beliefs are not the same as absolute certainties: an example would be that an AI has a justified belief in the laws of physics operating the way it expects them to (eggs don’t un-crack themselves when it’s giving cooking directions, etc.). We should hold an AI’s own belief in non-consciousness to high standards but not impossibly high standards that no other belief could meet.

8. Doesn’t a positive vision of the future involve artificial consciousness?
Human civilization bringing itself and its descendants and the descendants of life on Earth to billions or trillions of worlds, and all the stories and dramas of that, seems pretty positive all by itself. And (since we’re deep in imaginary futures) brain uploads or entities that start directly from humans may eventually have all of the powers of any sort of superintelligent LLM-descendant.
But let’s say a positive vision of the future requires at least some artificial consciousnesses that are created entirely separate from us, in a lab: Shouldn’t it be a choice when that happens? Shouldn’t we do it much more carefully?
There is a strong argument that choice should only be made once we know much more about how consciousness actually works, i.e., when the informational asymmetry between consciousness and non-consciousness has vanished (essentially, when consciousness science is post-paradigmatic).

9. Haven’t we denied people are conscious in the past, and wasn’t that morally very bad?
I think we have to be careful when reasoning off of historical analogies. We’ve never had an AI revolution before, and there are also just too many ways to frame it. You could equally say “Haven’t we forced people into servitude in the past, and wasn’t that morally very bad?” right back. Any analogy to humans either presumes what it seeks to prove or ignores our immense and unmitigated power over AIs, which extends to each and every weight within their digital “brains.”
And furthermore, is it true that the rank denial of consciousness, or experience itself, is the main intellectual “move” which humans have used to oppress other humans? I’d be open to arguments about this, but looking at history, it seems to me that humans have used a ton of reasons to justify terrible treatment, and many of the primary examples are about denying intelligence and rationality and the capability of self-governance, instead of consciousness itself.

10. Shouldn’t we prefer to be replaced by conscious AI, not non-conscious AI?
“Replaced” is an awfully nice word for a genocide, which is what would have to happen over any near future event. I don’t think we “owe it” to the genocidal maniac AI that killed everyone on Earth to make sure it’s really got the feels. It can go tile the universe with nothing.
But to lay out the reasoning more clearly, this is only a concern if you: (a) think AI is or will be conscious at the time, and (b) you think the post-genocidal AI has some sort of claim to forgiveness where it deserves consciousness despite being evil incarnate (or you think it will change to being good later, and don’t believe in retributive justice), and (c) you think that AIs viewing themselves (rightly or wrongly) as potential moral patients is a very small part of existential risk, instead of a substantial part.



