Anthropic's Chris Olah says AI must be guided from outside Big Tech

Anthropic cofounder Chris Olah said on Monday that the development of artificial intelligence cannot be left solely to technology companies, urging greater oversight from religious leaders, governments and civil society.

Speaking at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large ‌scale".

"If that happens, ⁠supporting those ⁠displaced will be a moral imperative of historic proportions," he said, sitting alongside the pope.


He ​added that companies like his operated under strong commercial, geopolitical and personal pressures that can ​be at odds with the broader interests of society.

"Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the ​right thing," he said, adding that even well-intentioned ⁠researchers remain ‌influenced by those forces.

Olah said this made outside scrutiny essential.

Anthropic ​is a ​U.S.-based company that produces the Claude AI tools. It has ⁠clashed with President Donald Trump's administration, notably by insisting on ​guardrails restricting how its models can be used for ​military purposes such as targeting weapons autonomously or domestic surveillance.

Olah welcomed the Church's engagement with the rapidly developing technology, saying the ethical questions raised by AI extended far beyond engineering.

"The questions raised by AI are bigger than the AI research community," he said and called for "earnest, thoughtful critics" who could challenge companies ‌and help steer the creation of powerful new systems in a positive direction.

Olah highlighted three areas he said required urgent attention -- ​the risk of ​widespread job losses, ⁠the need to ensure that AI benefits are extended worldwide, and the unresolved question of how to interpret increasingly complex and sometimes opaque system behaviour.

"AI development is ​concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally?" Olah said.

Monday's event marked an unusual convergence between the technology sector and the Catholic Church, which has sought to position itself as a moral voice on the implications of rapid advances in AI.