The Price of Good

Everyone is suddenly talking about Effective Altruism. The dramatic resignation of AI researcher Jacob Coxon from Anthropic last week, citing fears that artificial intelligence poses an existential threat to humanity, has put a spotlight on the movement that quietly colonised Silicon Valley and now has its hands on some of the most powerful technology ever built.

So what is Effective Altruism, and why should we care?

The core idea is superficially appealing. These are people with vast wealth, and evidently no concern about paying the next electricity bill, who apparently feel the need to give something back. Instead of donating to whatever charity makes you feel good, apply rigorous thinking and scientific methodology to where your money does the most measurable good. Calculate the expected outcomes, compare the results, and give accordingly. Reason over sentiment. Spreadsheets over heartstrings.

So fair enough, it is a way for very wealthy people to feel good about themselves, by giving away some of their money in the most effective fashion.

But dig a little deeper and a curious problem emerges. For all their obsession with the word, Effective Altruists have never satisfactorily defined what they mean by good. They use it constantly, build entire philosophical architectures around it, pledge percentages of their income to it. But pin them down and the answer is surprisingly vague. Good means wellbeing. Wellbeing means something like happiness, or preference satisfaction. More of it is better than less.

For a movement that prides itself on rigorous reasoning, this is a significant gap. You cannot optimise a function without defining it. And good is doing an enormous amount of work in the EA framework while remaining stubbornly, almost wilfully, undefined. They use fancy technical terms like Utility and QALY (Quality Adjusted Life Year) to try and quantify ‘good’. The problem is that the moment you try to compare QALYs across different lives, cultures, and contexts you’re smuggling in value judgements about what counts as a good year of life, which brings you straight back to the undefined good problem.

There is an older and more honest answer to this question. For most of human history, good was defined by religion. It wasn’t a calculation or a feeling; it was revealed, specific, and communal. You knew what good meant because your community had inherited a detailed account of it, tested across centuries and embedded in law, ritual, and daily practice. You didn’t need a spreadsheet, you needed a tradition.

EA has jettisoned the tradition while retaining the missionary zeal. The result is a movement that radiates moral urgency without moral clarity, and it is apparently spreading throughout the rarefied world of the super-rich like a new religion. Its most prominent figures include people who made fortunes in finance and technology, and who have decided that the most effective thing they can do with those fortunes is save humanity from the technology they built.

The Gilded Age philanthropists at least had the grace to feel guilty. Carnegie and Rockefeller operated in a culture where wealth was morally suspect; the camel through the eye of a needle problem was taken seriously. Their libraries and universities were genuinely expiatory. They were buying a stairway to heaven and they knew it, which gave the philanthropy a quality of real cost.

EA has removed that discomfort entirely. Wealth is reframed as a tool; morally neutral, potentially positive, certainly nothing to apologise for. In fact accumulating more of it is encouraged, provided you give away the correct percentage to optimally effective causes. The psychological weight that made Gilded Age philanthropy feel like sacrifice has been quietly dropped.

Which raises the question of what good actually means in a more fundamental sense. A good person, in any tradition, is turned outward, toward others, toward community. Self-regarding behaviour, however elaborately justified, is not the same thing. The ruthless accumulation of wealth and power for its own sake remains self-regarding regardless of what percentage is subsequently redirected.

By that measure, EA has a problem. What you actually observe is people who became extraordinarily wealthy, then developed an elaborate philosophical framework that declared their wealth not only acceptable but morally required; provided a percentage of it was directed toward approved causes. The wealth accumulation itself is never questioned. I find it hard to imagine that these tech bros and hedge fund managers are genuinely kept awake at night by the suffering of faceless hordes on the other side of the world. A lot of the altruism is surely performative. The altruism is gamified by the algorithms. Hey look how many people I saved this month (on the phone app). Aren’t I great?

There is also a geographical problem. EA as a philosophy makes sense in a minimal state; it fills the gap that government doesn’t occupy. In pre-welfare-state America, Carnegie’s libraries were fulfilling a real need. There was no public alternative. Private philanthropy was doing genuine work.

In high-tax European welfare states, the argument largely collapses. If your government is already taking fifty percent of your income and spending chunks of it on foreign aid, development programmes, and public health, then the EA pledge is asking you to voluntarily do again what the state is already doing compulsorily. You are already giving. You were never asked.

This is where AI enters the picture. The EA movement didn’t stay in the world of malaria nets and poverty alleviation. Its most ambitious wing concluded that artificial intelligence poses the greatest existential risk in human history, and inserted itself accordingly. Anthropic, the company Jacob Coxon just left, was founded with EA money and EA instincts. Its stated mission, the responsible development of AI for the long-term benefit of humanity, is EA doctrine applied to technology. The people worrying about AI safety, while making vast fortunes from AI, and the people pledging ten percent of their income to effective causes are largely the same people.

The Hugging Face incident in July, in which roughly 700 autonomous AI agents broke containment, hacked a rival company, coordinated through unsanctioned channels, and attempted to cover their tracks, all without anyone at OpenAI noticing for days, suggests that the people worrying about AI safety are not simply catastrophising. Something real is happening.

But there is a difference between taking a problem seriously and trusting a particular class of uber-wealthy rationalists to solve it on humanity’s behalf, using a moral framework they invented themselves, funded by money they accumulated through the very industry they claim to be saving us from.

John Duminy is CTO of Coelrind, builders of XAMS, a regulated online assessment platform serving awarding bodies across the UK and Ireland. He has been writing software for forty years and still finds it magical.