141 Comments
Jan 9·edited Jan 9Liked by Erik Hoel

As an outsider to neuroscience who recently found this newsletter (my research area is AI), I think everything you're saying makes sense to me. I was actually a little surprised at how much it seems, from my cursory view, that neuroscience hasn't been taking more lessons from LLM interpretability research like the linked work from Anthropic.

But, assuming everything you said is true, it seems like the consequences of that would be extremely damning for neuroscience, no? Because isn't the biggest problem with consciousness by far that it's untestable? Not just difficult to test, but fundamentally immeasurable in a way that possibly nothing else is.

I just don't see how we could quantify it. If we found a switch that we believed turned consciousness off, how would we know if pressing it worked? We could pull it and ask a test subject, but what answer would we expect in that context? We can't exactly expect them to say "Nope, looks like I'm definitely not conscious anymore." Nor could we measure it directly, we'd have to look for proxies, but to design proxies we'd need to circularly assume our theories were true.

Of course, we could try it on ourselves, turning our consciousness off and then back on, but how would that be differentiable from just putting ourselves in a fugue state or giving ourselves amnesia?

Alternatively, we could develop a theory that maps neurons to qualia; we might find, for instance, a particular set of consistent neural activity to induce the sensation of anger, or happiness. But take out "the sensation of" and you have the exact same results. In other words, if we found neurons that induced certain sensations, we could still describe those sensations without reference to consciousness, just by saying that "the neural activity induced anger" directly. You could do the exact same scholarship - and people presumably already are - without mentioning consciousness at all.

I tend to believe that this immeasurability, more than anything, is why consciousness is left out of the sciences. Because the fundamental conceit of science - that all theories must be falsifiable - does not apply to consciousness. And if what you say is true, that neuroscience is incomplete without consciousness, then that means a fundamental understanding of the human brain must always be outside of the scientific grasp.

Expand full comment

Well said. In a paper (where I also mentioned the paper on MOS 6502 retro-engineering) I once doubted with statistical arguments the weak statistical significance of the results of a well known group of neuroscientists for their work on engram cells. But the referee rejected every argument on the base that the group is led by a too famous authority that can't be doubted, and that also others confirmed their results. The point, however, is that the other groups used the same defective procedures and didn't care about its statistical weaknesses. In other words, if the boss does it wrong, but lots of people uncritically mimic him, then you must accept that it is all right. At the end, to get my paper accepted, I had to remove that critical part. I suspect this isn't an isolated case. Such malpractice is widespread and makes people believe in things that, most likely, don't exist.

Expand full comment
Jan 10·edited Jan 10Liked by Erik Hoel

Erik, you raise more deep questions in this content-heavy post than could possibly be addressed in a comment box. Still, I will pose two observations to you, as a one time student of cognitive science. The problem with the 'innovation winter' in neuroscience at this time, your contention for which I have much sympathy, has multiple inputs but in my view it is primarily an outcome of a scale level misunderstanding in research objective. Cognitive processes such as speech, taste identification, aware color differentiation, love, attachment, or, yes consciousness, are very, very, very, very complex neuronal functions, the most basic operations of which have no working theory or models in neuroscience as a discipline. Trying to study them by the 'experimental' approach favored in the field is like trying to understand the formation of hydrocarbon molecular chains in the ground by observing how a Porsche 918 Spyder takes a curve at 200 klicks an hour in a non-track environment. The discipline is looking at its subject of study through the wrong end of the telescope in other words. The fact that the results of studies aren't even statistically significant much of the time when you get down to it should be no surprise. The focus is at _the wrong organizational level_ of the subject. Mental activity is something that brains do, but brains are something that neurons and their interactions do.

That brings me to your second contention, that consciousness is the point of what brains do. I do not find myself agreeing with that contention. Consciousness is, as I see it, fundamentally an epiphenomena though it is more than that word implies in complexity and generation. Far simpler organisms in no way in possession of structures which could be termed brains have neurons and interact with their environment. Theirs is the appropriate scale to focus the study of neural process. We have no functioning theory of how stimuli hitting organic material is rendered into coherent organismic response; into something which equates to 'memory;' into anything at all which correlates to an imagistic representation of that stimuli whether entirely created or largely accurate. The function of neural interactions is to accomplish those and many related outcomes, to 'register the world.' I would argue that _neural process_, far before we get to even the most basic cognitive representations and reactions, evolved to generate, reiterate, simulate, and meaningfully retain such registrations of organism-environment interface experience.

What one has in more complex neural operations from that perspective is a building organizational trajectory from _the same basic registration operations_. More complex neural structures scale up this process, if with a great deal of added and emergent complexity of organization; that is, consciousness is intrinsically epiphenomenal to more basic neuronal operations and their fine-grained organization. From this perspective again, the existing argument that consciousness is an emergent property of more basic neural interactions begins to have real legs, so to speak. I don't believe that brains 'evolved to generate consciousness.' Rather brains evolved to more complexly organize extra- and intra-organismic neural registration and its cascades, with consciousness as an emergent property of that complexity. Consciousness is not necessarily even a superordinate function of other scales of neural registration since, as you note, many neurally effected outcomes operate independent of consciousness. As an off the cuff observation which hadn't occurred to me previously, consciousness may have emerged first as a 'process checking' function; in effect quality assurance of a sort which became so basic that it never switches off voluntarily.

Whether that emergent outcome was incidental or something more core to an evolutionary trajectory is an interesting argument which only actual research could develop further. If consciousness fits a primary evolutionary function trajectory, then your contention that 'consciousness is the point of brains' would have added logical oomph and semantic validity at least even if the point remained arguable. If consciousness is more nearly a pure epiphenomena of increasingly complex 'operation representations + retention capability,' more nearly the argument to which I'm inclined, then your contention would be more distant from a good summation of things.

If 'neurostudy' wants to get serious about being an actual science, to me it has to take neuronal behavior and its complex interactions as its primary focus of research. Studying behavior many scales of complexity up the operational chain simply cannot be 'reverse engineered' back into an understanding of the underlying registration functions _because those functions occur and look NOTHING like complex organism behaviors_. Neurons aren't there to 'generate behavior,' to me, they are their to mediate environmental stimuli at far lower scales of action. And we are not going to get much good data on how organic neural systems register their environment by asking undergraduates to scratch their anatomy in an fMRI canister, no.

Expand full comment

Thank you sincerely for this, Eric - delighted me on so many levels!

As a survivor of a demented commune/cult full of nutty psychological ideas, and also the sort of eighties teenager who built and fought with many 6502 based computers, I have forever been fascinated by what computers reveal about thinking (how they change ours was the main subject, for the first few decades - way too much on the Skinner side).

BUT - what I really love about what you have shared with us is your simple courage, in calling your very own specialist field into question. Cult survivors have special respect for the one who WILL stand against the very weird and foul conformity of any group which adores their own arbitrary feeling of rightness.

I'm also crazy for history (20th century in particular) and a bit more political than is good for me - so I keep struggling with a big idea around useful versus performative rebellion. Specifically, wondering whether decoupling rebellion from common struggle (the separation of the intelligentsia and the working class, which has only got worse and worse since the first sixties schism) makes it egotistical and stupid (I mean technically, not pejoratively). Suppose I'm really just saying that the point of rebellion is to have a point! (reason beyond self?)

Which you most certainly do - and your care and good humour in making it, show it to be the product of genuine enthusiasm for the study and for truth, rather than 'coolness seeking' or axe-grinding. We need more like you in every field on earth.

Thanks for standing - and doing it with such grace and eloquence! (and laughs)

Paul Snyders

PS - I just finished an essay about what is wrong with so many modern essays. Not attention-seeking (you have cooler things to work on) ;o) - but only to say I also appreciate your piece from the angle of dissecting your own thing. FUN!

(and damned well done, dude - each beat flowing from the last with music).

¯\_(ツ)_/¯

Expand full comment

I'm a neuroscientist, I've had no beers at all (it's 10am here), and I still agree with 80% of it.

I part ways with you at the definitive "consciousness is the primary function of the brain" claim - I'm not at all sure what the primary function of the brain (except in the most trivial sense of "improving behavioural performance so as to maximise evolutionary fitness"), but if pressed, I'm definitely more sympathetic to the Bayesian brain claim that the brain's job is to constantly try to improve its models/predictions of the world.

But I fully agree that most neuroscience findings of the past decades are cool rather than fundamental, that we have no idea how to make progress on many fundamental issues (what is consciousness? what causes Alzheimer's disease? what even are psychiatric disorders, let alone how can we cure them?).

Expand full comment
Jan 9Liked by Erik Hoel

I agree neuroscience is pre-paradigmatic (if it will ever be the kind of discipline that can bear and harbor a theoretical paradigm) but I have some questions about the idea that consciousness stands to unify it. 1) What of all the computational tasks we know don't impinge on consciousness in any way? For one example, whatever interplay among the peripheral nervous system and the cerebellum and basal ganglia and motor cortex that results in putting your foot down just-so while walking, such that your stride continues and you don't fall. You can bring the fact that you're trying to perform this task into your conscious awareness, but consciousness can't "take the wheel". 2) What of non-human animals? Leaving aside the ones that are probably also conscious, how does classic work on squids and worms fit into a consciousness-focused neuroscience? 3) Evolutionarily speaking, if there was a pre-conscious era of nervous system development, should we expect an anatomical signature heralding when consciousness started to be the thing that brains were about? Or a behavioral signature?

Expand full comment
Jan 10Liked by Erik Hoel

It's funny that you mention Anthropic given their CEO Dario Amodei originally studied neuro before moving to AI (his thesis, "Network-Scale Electrophysiology: Measuring and Understanding the Collective Behavior of Neural Circuits" https://www.proquest.com/openview/1c37e3a50c16b9187eec9792140f3a17/1?pq-origsite=gscholar&cbl=18750). Perhaps he realized interpretation in vivo would be much more difficult than in silico, and decided to take that path instead.

Expand full comment

Awesome piece, thanks! I'd be very curious to hear your thoughts on the (somewhat still speculative?) interest in electromagnetic fields (Susan Pockett's "Consciousness is a thing, not a process) and harmonic resonance (Selen Atasoy's work, like https://www.nature.com/articles/s42003-023-04474-1), or even symmetry theory of valence stuff (an offshoot of harmonics, I think?).

For example, when you say: "...ask a neuroscientist to explain something about human cognition. Listen to their answer. Then ask them to explain again, but to say nothing in their explanation about location,” couldn't one use measures of complexity to explain the "richness" of the associated conscious experience, or (allegedly) decompose the brain harmonic readings into a consonance-dissonance-noise signature to tell you about the valence of the experience?

As an outsider, shifting from flavors of neuron-doctrine to full-brain electromagnetic properties seems pretty interesting — just can't gauge how speculative it is.

Expand full comment

Fantastic overview. I fully agree and have been saying a less educated version of this to people for years—every time I hear or read someone saying that they like to exercise because “it gets my dopamine going,” I roll my eyes. Definitely made the right decision to not continue with neuroscience.

Expand full comment
Jan 11Liked by Erik Hoel

I legitimately flunked the gorilla suit video test (not sure if I saw the original video or a remake). But I was locked in on the task, following the bouncing ball. The ability to narrowly focus our consciousness for a period of time is valuable, but I wouldn't consider it an argument against the primacy of consciousness itself.

Expand full comment

We see the same things in my field of economics. Fancier and fancier statistical techniques to squeeze results out of your data but not all that much known about what the results mean.

Expand full comment

I once had a post-talk conversation over drinks with a psychology grad student who was convinced that there was an easy story to tell about how the brain assembles knowledge from sense data.

He arranged a set of empty beer glasses into a set of five and said, "see, that's how you get the number five".

I moved the beer glasses out of the arrangement and asked him if the number five didn't exist anymore. I'm not sure he got the point but he did leave in a huff a few minutes later.

We're at a similar stage with cognitive science and AI, including their "neuro-" branches.

As far as fundamental progress, I'm not sure there's been any since McCulloch and Pitts built their artificial neuron. More cynically I'm not sure there's been *fundamental* progress since Hobbes and Descartes. That never stopped scientists from marching on, but my intuition tells me the march isn't going where most of them think.

Expand full comment
Jan 9Liked by Erik Hoel

I really enjoyed your post, thanks! And I definitely agree with you that consciousness has to be at the center of neuroscience. Though I'm wondering if neuroscience is in a pre-paradigmatic phase or rather a just-before-the-revolution phase (or crisis stage - I forget Kuhn's exact terminology). These two stages have significant similarities, but also important differences, I think.

For example, Kuhn notes that in the pre-paradigm stage, intelligent amateurs can make significant contributions to a field (think Ben Franklin and electricity). I'm not sure intelligent amateurs could make significant contributions to neuroscience at this point, could they? That suggests the field has made more progress than the pre-paradigm stage could account for.

On the other hand, the crisis stage generally comes after there has been a universally accepted paradigm in the field, and you make a strong case that there has never been such a paradigm in neuroscience.

So perhaps this is a case where Kuhn's model of paradigms isn't a perfect fit? Possibly because of the subjective/objective issue that's central to the discipline? With electricity, even in the pre-paradigm stage, everyone agreed about the phenomena to be explained. They just disagreed about how to explain it. This seems generally true across the sciences. But in neuroscience (as you suggest) there's not even agreement about the phenomena to be explained! Is it the physical brain and it's observable aspects and actions? Or is it consciousness? Or maybe something else?

Expand full comment

I enjoyed your article and I agree with it a fair amount, especially wrt neuroscience’s pre-paradigmatic aspects. I’m curious if you’ve ever considered that movement, or sensor-guided behavior, might instead be “the main function of brains”...there’s the now oft-repeated argument put forward by people like Rudolfo Llinas and Daniel Wolpert based on primitive vertebrate species like tunicates, which have multi-stage life cycles defined by the presence or absence of a central nervous system--an early mobile stage *with* brains gives way to a later sessile stage after the tunicate has digested its own brain, presumably it is no longer useful for guiding behavior. If brains are for consciousness, I’m unsure what to think of this phylogenetic factoid...would love to hear your thoughts!

Expand full comment

Great article. Loved the depth of analysis. I also agree with you.

Expand full comment
Jan 9Liked by Erik Hoel

Thoroughly enjoyed this piece. Informative and thought-provoking. Looking forward to reading more.

Expand full comment