A curious thing happened in late March: more than a dozen religious thinkers and representatives were invited to a discussion organised by Anthropic, the makers of the Claude chatbot, which recently revealed plans to set up an office space for 800 people in central London’s Knowledge Quarter. The religious invitees who included Christian scholars from universities and think tanks, then later expanded to other religions, were gathered by the AI firm to try and help them provide the system with moral guidance. Anthropic hoped that organised religion could provide some form of support that could help shape the constitution, a document by which Anthropic’s AI model is meant to abide.
It’s not the only time that religion and cutting-edge technology have combined. In late May, Pope Leo XIV released a 43,000-word treatise on AI saying that the technology ought to be disarmed, equating it with the Tower of Babel, and calling on politicians to better regulate AI systems. “To disarm does not mean rejecting technology, but preventing it from dominating humanity,” the Pope said in an address at the Vatican.
Alongside Leo at the Vatican was Christopher Olah, a co-founder of Anthropic, who said that “what character we choose [for AI], how it interacts with the world, how it ought to interact with the world” are all “more clearly questions for the humanities, for religion, for philosophy, for society at large”.
It was a massive moment that astounded religious schools and AI watchers alike. “One company effectively has the ear of arguably the most influential religious figure in the world,” says Peter Phillips, director of research at CODEC, the Centre for Digital Theology at Durham University. But it begs the question: why is organised religion and at least one leading AI lab becoming so intertwined?
Has Silicon Valley lost control?
Phillips doesn’t think the tech industry’s turn to theology is necessarily cynical. He thinks it may be a sign that AI companies are worried they have created something they do not fully understand. “My hunch would be that they know something different is happening than they expected,” he says, “and that they don’t know how to deal with it, and therefore theology comes in as a possibility.”
The fear within the AI labs is that AI, without a strong moral grounding, could quickly spool out in ways that are harmful to society. Anthropic severely throttled access to its latest model, called Mythos, last month due to fears it could be used for nefarious means to hack IT systems. If AI becomes sentient, the concern is that it could do those things autonomously if not given strong direction not to.
Whether the impulse is sincere or not, the impact is definitely political. The Pope is one of the biggest figures in society, and has the ears of world leaders — turning theology from a source of reflection into a source of legitimacy — and, some argue, is a shield against criticism. And given that Anthropic plans to expand its presence in London, it seems likely that some of those who help formulate Claude’s moral thinking will be based in the UK. “London is already one of our most important research and commercial hubs outside the US,” said Pip White, Anthropic’s head of EMEA north, in a statement announcing the firm’s office move.
Organised religion is handy for that because it’s spent centuries thinking about questions Silicon Valley is only now stumbling into, including things like what counts as personhood, what makes a thing good, and who gets to decide? “I think the religious should be talking about putting parameters around the AIs that actually encourage them to be good, rather than encourage them to be efficient,” says Phillips. The aim, he argues, should be the “common good”, not more capable systems or more profitable ones.
Morality is more than just words
Not everyone is so convinced that the religious slant is a good thing. “I think some of the concern is that this stuff is now becoming a form of idolatry and idol worship,” says Catherine Flick, an AI ethicist at the University of Staffordshire. “When we see patterns that look anthropomorphic, that look like human traits, we generally ascribe them those sorts of traits,” Flick explains. And in an AI system that is ultimately a pattern-matching machine, that’s a worry. “If they’re getting tricked, what hope is there for the rest of us?” But the more AI companies talk about values, character and moral formation, the more they risk inviting users to think about their systems as human-like, if not human entirely.
Anthropic may want Claude to behave well, but Flick argues that Claude’s values document is different to actual moral understanding. “These are just words, right?” she says. “They don’t actually have any intent or meaning or understanding, or anything of those sorts for the model itself.”
Some have suggested that involving religion is a convenient way to try and excuse responsibility for the actions of a chatbot that they developed and released to the public before baking in answers to some of these more thorny questions. The risk is that the future moral architecture of AI is shaped through private conversations between a handful of powerful companies and hand-picked moral authorities. But Flick worries there’s also risk that doing so overlooks the real problem.
“The harms are not going to be solved by putting more ethics or religion into it,” she says. “I don’t think you can fix that with just injecting values, because that means absolutely nothing.” The Pope’s words were a clarion call for action — but whether his views, heavily tinged by Anthropic’s input, are the best route of action is still uncertain.
