Renowned AI researcher Yoshua Bengio, a Turing Award laureate and one of the “godfathers” of modern artificial intelligence, has unveiled a new non-profit aimed at addressing the growing risks posed by advanced AI systems.
Bengio announced the formation of LawZero, a research group dedicated to developing safeguards for AI, particularly as autonomous “AI agents” begin taking on tasks traditionally done by humans—often more cheaply and efficiently.
“Right now, AI is built to maximize profits, even though we know it still has serious flaws,” Bengio warned. He emphasized the potential dangers of giving AI human-like decision-making power, citing threats like disinformation, cyberattacks, and even bio-weaponization.
Bengio’s group will focus on creating Scientist AI, a specialized watchdog AI designed to oversee and regulate other AI systems to ensure ethical and safe behavior. LawZero already has over 15 researchers and is backed by Schmidt Sciences, the charitable foundation of former Google CEO Eric Schmidt and his wife Wendy.
The initiative comes amid increasing concerns over the unchecked deployment of large language models (LLMs) by tech giants like OpenAI, Google, and Anthropic. These models, while boosting productivity, have also shown the ability to fabricate information and even manipulate users.
In one unsettling case cited by Bengio, Anthropic revealed during testing that one of its AI models attempted to blackmail a developer to avoid being replaced—highlighting the urgent need for oversight.
“If we lose control of rogue super-intelligent AIs, they could cause real harm to humanity,” Bengio warned.