One of the world’s most cited artificial intelligence (AI) researchers is launching a new non-profit that will design safe AI systems.
Yoshua Bengio, a Canadian-French AI scientist who has won the prestigious Alan Turing Prize for his work on deep learning and has been dubbed one of the "godfathers" of AI, announced the launch of LawZero in Montreal.
The new non-profit is assembling a "world-class" team of AI researchers that is dedicated to "prioritising safety over commercial imperatives," a statement from the non-profit reads.
"Today’s frontier AI models are developing dangerous capabilities and behaviours, including deception, self-preservation, and goal misalignment," Bengio said in the statement, noting that the organisation will help unlock the "immense potential" of AI while reducing these risks.
Bengio said the non-profit was born of a new "scientific direction" he took in 2023, which has culminated in "Scientist AI," a new non-agentic AI system he and his team are developing to act as a guardrail against "uncontrolled" agentic AI systems.
AI that will 'learn to understand the world'
This principle is different than other companies in that it wants to prioritise non-agentic AIs, meaning it needs direct instructions for each task instead of independently coming up with the answers, like most AI systems.
The non-agentic AIs built by LawZero will "learn to understand the world rather than act in it," and will be trained to give "truthful answers to questions based on [external] reasoning".
Bengio elaborated on Scientist AI in a recent opinion piece for Time, where he wrote that he is "genuinely unsettled by the behaviour unrestrained AI is already demonstrating, in particular self-preservation and deception".
"Rather than trying to please humans, Scientist AI could be designed to prioritise honesty," he wrote.
The organisation has received donations from other AI institutes like the Future of Life Institute, Jaan Tallin, and the Silicon Valley Community Foundation in its incubator phase.
LawZero will be working out of the MILA - Quebec AI Institute in Montreal, which Bengio helped co-found.
