44 Comments
Aug 23, 2022·edited Aug 23, 2022Liked by Erik Hoel

The fundamental error in utilitarianism, and in EA it seems from your description of it, is that is conflates suffering with evil. Suffering is not evil. Suffering is an inherent feature of life. Suffering is information. Without suffering we would all die very quickly, probably by forgetting to eat.

Causing suffering is evil, because it is cruelty.

Ignoring preventable suffering is evil because it is indifference.

But setting yourself up to run the world and dictate how everyone else should live because you believe that you have the calculus to mathematically minimize suffering is also evil because it is tyranny.

Holocausts are evil because they are cruel. Stubbed toes are not evil because they are information. (Put your shoes on!)

Expand full comment
author

This is a great way of putting it

Expand full comment

Strong second re: suffering.

Suffering is kind of a paradox: most people agree you should work to eliminate almost any specific example of it...but to eradicate it in total seems crazy. To a person, we regard past suffering as instrumental, valuable, and indeed a core component of our being.

I wonder if perhaps there's a major correlation here in terms of parenthood (or lack thereof). I feel like suffering being an instrumental part of development is something every parent has to confront, and which becomes obvious to every parent.

Expand full comment

Certainly, part of growing up is learning to bear suffering and to act resolutely and justly while suffering. Whether this permits a parent to impose suffering on a child as a means of developing this capacity is an interesting question. We need a line that permits making them eat their vegetables and do their chores while forbidding random beatings. It can be a hard line to draw, though perhaps there is a useful rule of thumb, which is that the parent should never enjoy it.

Expand full comment

Indeed. It seems not only to be a tough question to answer, but a tough question to answer *that potentially has a different answer every time it is asked*. The only real takeaway, I think, is simply to establish that suffering is *potent*, in the same way fire is. But neither fire nor suffering are simply bad, full stop. I can, I think, imagine a child who has never suffered. And I imagine it would be a monster.

Expand full comment

" we regard past suffering as instrumental, valuable, and indeed a core component of our being. " - not to be overly cynical but I think these are largely fanciful stories we tell ourselves to paper over the pain. So, for any trajectory of a persons development which involves suffering, I bet you can find trajectories where the vast majority of the suffering wasn't there yet their development was just as good, if not better. Maybe a tiny bit is useful somehow, but I think we could eliminate 99% of suffering with no drawback and only positive effects.

Expand full comment

The number of undisciplined and ineffectual children now graduating from our universities would suggest otherwise.

But the larger point is that, while we have eliminated a great deal of the suffering that used to plague human lives, the prospect of eliminating 99% of the remaining suffering seems forlorn. Suffering, after all, included such trials as unrequited love, obscurity, and loneliness. More profoundly, purposelessness seem to bring a particular kind of suffering all its own, and yet without challenge there is no purpose. And where there is challenge, there is suffering, or it is not a challenge. And consider how often people take on entirely unnecessary suffering just for the sake of entirely unnecessary challenges, such as pursuing Olympic medals.

Expand full comment

I don’t think I would want to live in a world where no one would voluntarily undertake an extreme discipline in order to achieve some reward, spiritual, psychic or physical.

Expand full comment

I'm sure sometimes they're stories we tell ourselves, but I think that's more than counterbalanced by the completely primal and animal way we react to suffering in the moment, which is: *stop stop please stop I'll do anything to make it stop*. Post-hoc narratives, after all, can also be used to amplify and prolong suffering. I'd say that's the more common outcome. It's certainly the more innate outcome: a four-year-old will react to a skinned knee with pure anguish. The natural human tendency is to overreact to and overplay our suffering, not to be thoughtful and positive about it.

There's also the question of whether there's really a difference between suffering teaching us a lesson, and simply deciding it did. Maybe we invent the lesson, but maybe that's self-fulfilling, too.

Expand full comment

Wouldn't you agree that curing disease and alleviating unnecessary suffering is extremely good, though? That's what utilitarianism says.

It's actually not clear if some pain/ suffering is necessary. In theory we could get information from a source that doesn't involve pain and suffering, for instance via a bright "light" in the brain. You can also consider gradients of bliss to send information -- see the writings of David Pearce.

Expand full comment

I would say that curing disease and alleviating suffering are good because they are acts of love.

Suffering is bad, in the sense that it is unpleasant, but it is not evil. It is a signal. It is an effective signal precisely because it is unpleasant. It motivates us to do something to remove the suffering, like pulling our hand out of the fire, for instance. A light in the brain would not motivate us in the same way.

Doing something to alleviate our own suffering is an act of self-preservation. Doing something to alleviate the suffering of others is an act of love.

Utilitarianism is wrong in that it focusses on the reduction of the signal. Yet there is a way to reduce the signal to zero instantly. Simply kill the patient. It is by far the most expeditions way to eliminate suffering -- and this is not a abstract example. People are really making this argument and acting on it.

Love, on the other hand, acknowledges that suffering is bad and that we should alleviate it where we can, but it recognizes that there are values that transcend the mere question of suffering. Love forbids us to kill to cure. Love forbids us to kill one person to harvest their organs to cure 10 others.

Suffering is endemic to the human condition. We could end all suffering by ending all life. But we don't because we hold life to be sacred. Unfortunately we are inclined to pick and choose which lives we hold sacred, because we often find more local utility in having some die. But at least we are still repulsed by the grim calculus of pure utilitarianism.

Expand full comment

Keep in mind that alleviating suffering for one may be imposing suffering on oneself or spreading it out to others.

Expand full comment

Causing suffering is teaching. We do it without guilt because our lives depend on it.

Of course causing suffering gratuitously is evil.

Expand full comment
Aug 23, 2022Liked by Erik Hoel

The diluted form of utilitarianism that makes the most sense to me, and which does still feel compatible with the general EA ethos, is one in which you don’t feel constrained by the results of the utilitarian calculus, but you should actually make the effort to do the math before deciding.

For example, before choosing where to give money to charity, I think it’s very much worthwhile to try and do some kind of calculations to compare them. This forces you to actually consider all the factors and identify your unknowns. But your decision should still be based, in the end, on what seems like the best choice, even if one has a higher expected value on paper.

This isn’t a formal philosophical way of looking at things, but it seems like it avoids the failure modes where you don’t do the math and end up donating to local causes that don’t need as much help as international ones AND the failure modes where you would trade massive numbers of stubbed toes for the Holocaust.

Expand full comment

Coincidentally Scott A also reviewed W MacAskill's book today. Having read both reviews one after the other I have more than a headful of philosophising about morality. In fact that particular behaviour, more than any other is deserving of the expression "pouring from the empty into the void", and perhaps that's my personal deep-rooted objection to EA, Utilitarianism, and to some extent the wider rationalist movement. It's a whole lot of head noise utterly divorced from a human scale, inter-personal considerations and any kind of emotional intelligence. It is unsurprising that one of its outputs is repugnant conclusions.

Oh, I very much enjoyed your review/follow-up. I preferred it to Scott's, which is about as much praise as I offer in an average week :)

Expand full comment
author

Thank you! And I think you're spot-on regarding some of the difficulties of getting people to swallow these ideas. I saw Scott's review of What We Owe The Future just as I was getting ready to send this one. I enjoyed Scott's review, especially the comparison to a "philosophical mugging," and don't mind synchronicity.

Expand full comment
Aug 23, 2022·edited Aug 23, 2022Liked by Erik Hoel

One of the biggest weaknesses, in my view, of Utilitarianism is the fact that given some assumptions about the "moral value" of a set of actions/events, anything is defendable. The fact that some of your interlocutors will insist on adding "epicycles" to reason away the various repugnant conclusions of a line of utilitarian thought belies this underlying fact. Give me a vector of utlis long enough, and I shall move the world.

That said, I think that there are useful tools which can come out of utilitarianism. In the finance world, the Black-Scholes model, while certainly "incorrect" in its underlying assumptions and predictions, is still an immensely useful model with which to understand the risks associated with optionality. Take the model to its extremes, however, and it quickly breaks down. I think of Utilitarianism in a similar vein. It's key assumptions are certainly wrong, but it is still a rather useful tool to have when examining the possible morality of different courses of action. You just need to understand that, as a simplistic model of a much messier reality, it cannot be taken as truth. The map is not the territory etc etc.

Expand full comment
Aug 23, 2022Liked by Erik Hoel

Something that came up in a lab meeting here today: exactly what is *really* going on, as a psychological matter, in the Utilitarianism of EA? (You gesture at an interest in this question this in your portrait of Lovecraftian Cape Cod.)

One thing in play is the role of explanatory values in moral reasoning -- meaning, what it is about moral explanations that makes one more satisfying than another. Virtue ethics (which you gesture at in your polar bear poem) is a very distinct mode from EA but it really fails to get traction with people these days, and it’s often dismissed as “aesthetics” (a Gen Z term of art I don’t quite grok).

Utility theory has the advantage of producing highly co-explanatory moral justifications. “X is moral for the same reason Y is moral”, even when X and Y are in very different domains, timescales, and contexts.

I don’t know what is behind the shift towards increasingly extreme versions of utility theory (MacAskill saying basically the RC is probably right is insane, and Parfit rolls in his grave!) But it might lie in the structural features of the underlying moral cognition.

I see EA as having a really profound attraction for young people. Mainstream media is well behind the curve on this -- I started getting EA-interested students five years ago. It’s quite impressive.

Expand full comment

Thanks for both essays good sir, I spent an afternoon remembering why I never was satisfied by utilitarian arguments in my philosophy classes. Your mention about California being a utopian land that breeds these ideas particularly resonated. There’s something particularly nefarious about the continued American obsession with Manifest Destiny and it’s more contemporary technocratic/consumer cultural form, which one could easily argue has been the root cause for the very same problems wealthy EA folks out in California think they have a duty to fix. Colonialism takes on many forms, indeed.

Expand full comment

Thanks for writing this, and your previous EA piece. I have long felt that many of the EA arguments, especially those comparing dust motes and murders, were obviously wrong. The unfortunate thing about the obvious is that it’s much harder to articulate a Why.

Expand full comment

One of the more troubling and little-noted features of these grand moralizing theories is the unspoken subject addressed by the so-called moral obligations.

Throughout these writings, there is a persistent "we" appearing in many normative sentences of the form:

"We ought to..."

Fill out the ellipsis with your preferred act, event, or state of affairs.

Whenever I see this in a moral argument, I find myself opting for the Tonto response.

"Who is 'we', kemosabe?"

That's not intended as a flippant rejoinder. Whether it be traditional Utilitarianism, its updated cousins based in rules or alternative states besides pleasure/suffering, or EA itself (whatever that is after the endless gerrymandering), the "we" is persistent.

Who or what is this entity?

It's never made explicit, which I believe is by design.

Depending on your moral imperative, it can be you (the individual human being with agency), or it can be a nebulous collective agent, such as an ethnic group, a nation state, or one of those trans-national organizations that do much of the heavy lifting these days.

Nice as it is to speak for all and everyone from that View From Nowhere, it's not going to convince everyone as a *moral point of view*, and for reasons raised in this article.

There is a real and important difference between having a map of the national park and making the 3-day hike through it. But few of these abstract moral [sic] theories seem to notice or care about that omission.

If the humans hiking through the forest in real time enter into the equation at all, it's only in the form of proxy states like "pleasure" or "suffering". A real and healthy human person is not moved by a single source of moral reasons or considerations, not least of which because a real and healthy human person is more than a single dimension of psychological content.

MacIntyre never wrote about EA directly, but his attacks on the figures of the Manager and the Therapist are just as appropriate. EA is management, not morality.

Expand full comment
Aug 24, 2022Liked by Erik Hoel

Galileo's Paradox and ignorance of set theory : why 'long termism' is ncompatible with 'Mound o' dirt' Utilitarianism.

Infinite Sets are a source of interesting paradoxes. "What is larger," wondered Galileo Galilei in _Two New Sciences_, published in 1638, "the set of all positive numbers (1,2,3,4 ...) or the set of all positive squares (1,4,9,16 ...)?"

For some people the answer is obvious. The set of all squares are contained in the set of all numbers, therefore the set of all numbers must be larger. But others reason that because every number is the root of some square, the set of all numbers is equal to the set of all squares. Galileo concluded that the totality of all numbers is infinite, that the number of squares is infinite, and that the

number of their roots is infinite; neither is the number of squares less than the totality of all the numbers, nor the latter greater than the former; and finally the attributes "equal," "greater," and "less," are not applicable to infinite, but only to finite, quantities.

See 'Galileo's Paradox' https://en.wikipedia.org/wiki/Galileo%27s_paradox .

Which brings us to the utilitarian's favourite hobby horse -- trolley problems. Each person tied to the tracks represents an infinite set of possibilities of things that that person could do, in the future, if the train doesn't run over them. Because I am a long termist, I care about depriving the future of all these possibilities.

The people who think that killing 5 people is 5 times as bad as killing only 1, are reasoning in the same way as the people who believe that the set of all numbers must be larger than the set of all squares. They are flipping the train switches because the results align with their ignorance of the properties

of infinite sets. A rudimentary course in set theory will remedy this problem. (And, as an added bonus ... set theory is fun.)

'Mound o' Dirt' Utilitarianism is thus dead in the water. Infinite sets are not fungible, and neither is moral worth.

Expand full comment
Aug 24, 2022Liked by Erik Hoel

I love your articles, Erik.

Completely agree with your disgust toward utilitarianism.

Since when did it become thinkable to harm anyone for the "greater good"?

As Steve Pinker points out, the greatest atrocities in history were claimed to be for the "greater good."

Expand full comment
Aug 23, 2022Liked by Erik Hoel

I can’t thank you enough for writing these kinds of pieces. I’m not sure anyone else exists in this same kind of niche; simultaneously aware and up-to-date on the kind of technological futurism and Kurzweilian philosophy the likes of SA, Hanson, Eliezer, and others traffic in, while being uniquely skeptical as to their effects.

I fully believe that, even though consciousness will not be a “solved problem” for AI, some level of AGI will nonetheless exist that has incredible universal ability and inevitably, incredible capacity to deviate from its programmers’ intentions. Rather than accept that and become a booster for its creation, the idea fills me with a kind of dread that I can’t shake.

Thank you for intelligently vocalizing some of these concerns in novel and interesting ways. Also: The Revelations was fantastic.

Expand full comment
author

Much appreciate the encouragement Maxwell, and so glad to hear about The Revelations!

Expand full comment

An additional counter to the serial killer surgeon "That all should go into the equation" response: even though a utilitarian may agree that the fear over this hypothetical surgeon should be part of the equation, they also are actively working to persuade people NOT to fear it in the first place. I'm not sure it's really a defense of the idea to say "don't worry, if people still hate the idea we'll factor that in."

In fact...is that even internally consistent? We're talking about almost infinite advances in util, people! Surely pushing through our short-term disgust with such things is well worth the upside in the long run. If anything, the cultural fear/pushback could be used as evidence that we must push even harder.

That, for me, is a big part of the problem: I have an inherent distrust of any idea that can use resistance and/or failure states as reasons to do even more, rather than reasons to do less (e.g.: "we're spending so much money on this social program and the results are poor, clearly that just means we need to spend even more").

Expand full comment
Sep 3, 2022Liked by Erik Hoel

Congratulation! - Not just me, but many others voted your review as the best https://astralcodexten.substack.com/p/your-book-review-the-dawn-of-everything?utm_source=substack&utm_medium=email

and I am very glad to see you as the author and winner. Very well done! Outstanding text! Standing ovations, indeed.

Expand full comment
author

Thank you so much Mark, it’s much appreciated. That contest was loads of fun to participate in.

Expand full comment

I'm wondering what you think of GiveWell? I like GiveWell's approach to charitable giving, and least until recently, I thought it was the most practical manifestation of effective altruism. It was disconcerting to see people talking about a "bednet phase" in the New Yorker review.

It seems like reccommending charities based on some rough but rigorous investigation into cost-effectiveness should be possible?

Expand full comment

Love the coda! I would agree that the environment we live in significantly shapes our worldview, so living in a place of perfect weather and abundant sunshine would give you a more optimistic outlook. Makes sense, then, why Californians are the way they are.

Expand full comment

Thoughtful piece about this book and the criticism is very fair, even though the main point and thesis of the book that we should be acting with a longterm view is 100% correct. We don't need utilitarianism at all to come to this conclusion and belief. Utilitarianism like any single philosophy has good and bad aspects to it. You have keenly identified so much of the negative here. Utilitarianism is cold, inhuman, detached, and "rational" and does not account for the lived human experience. As you point out it leads to horrifying conclusions when it is unfolded step-by-step according to its own tenets.

Any approach to solving the metacrises of the 21st century is going to require an approach that is integral and accounts for the full lived human experience, internal and external, as well as all of the best science and philosophy applied together.

One of our greatest failures as a species would be to not learn from the modern era in which pure science and pure reason were elevated as the only value. The modern era has given us so much valuable knowledge and many tools, but it must morally, ethically, and spiritually be integrated at the individual and societal level to be of use in advancing our species. We know enough now to know we can't utilize any single religion, discipline, or philosophy to solve our problems. To pretend otherwise would be naive.

Thanks for your excellent article.

Expand full comment