The growing debate over how quickly artificial intelligence should advance has gained another prominent voice, with Microsoft CEO Satya Nadella supporting stronger safeguards and greater human oversight for increasingly capable AI systems.
Nadella’s comments come amid a wider discussion in the technology industry about whether the development of advanced AI should be slowed or more carefully managed. Anthropic CEO Dario Amodei recently called for a more measured approach, warning that AI progress could eventually move faster than humanity’s ability to control powerful systems.
Similar concerns have previously been raised by OpenAI CEO Sam Altman, Tesla CEO Elon Musk and Google DeepMind CEO Demis Hassabis.
Satya Nadella Calls for Human Control Over Advanced AI
Nadella, who is also Microsoft’s Chairman, said the pursuit of highly advanced AI must remain focused on keeping these systems under human control.
In a post on X, he argued that developing superintelligent AI only makes sense if the technology ultimately benefits humanity and remains subject to meaningful human oversight.
His comments reflect Microsoft’s broader position that AI innovation should continue, but with safety and responsible development built into the process.
Microsoft CEO Supports ‘Embedded Evaluators’
One of the ideas Nadella specifically backed is the use of embedded evaluators to assess AI systems continuously.
The concept has also been supported by Amodei and Altman and involves building mechanisms into the AI development process that can continually evaluate the behaviour, capabilities and safety of increasingly advanced models.
Nadella welcomed research and deliberate efforts to make AI alignment a core design objective rather than something addressed only after systems become more powerful.
He also suggested that the industry needs to develop practical mechanisms capable of turning AI safety principles into real-world safeguards.
Nadella Wants AI Benefits to Be Widely Distributed
While supporting stronger safety measures, Nadella also stressed that AI development should not be unnecessarily restricted.
He called for the benefits of advanced AI to be expanded across countries, communities and businesses. According to Nadella, this requires an AI ecosystem where both closed and open-source models can develop and compete.
His comments highlight the balance technology companies are attempting to strike between rapid innovation, broad access and responsible AI development.
Businesses Should Maintain Control Over Their AI Knowledge
Nadella also highlighted the importance of giving companies greater control over their proprietary knowledge and AI systems.
He argued that organisations should be able to build their own continuous learning systems without becoming completely dependent on a single AI model provider.
Companies, according to Nadella’s approach, should have the ability to incorporate their own knowledge into models and control the resulting systems and model weights.
This could allow businesses to develop AI systems that are more closely aligned with their specific operations, data and requirements.
AI Governance Should Not Be Controlled by a Few Companies
Another major point raised by Nadella was the need for broader participation in AI governance.
He warned against allowing decisions about the future of artificial intelligence to be concentrated among a small group of companies or organisations.
Instead, he argued that governance should involve a wider range of stakeholders, including different countries, industries, academic institutions and other fields.
The idea reflects growing calls for international cooperation and shared standards as AI systems become increasingly powerful.
Microsoft’s AI Safety and Governance Approach
Nadella outlined several elements of Microsoft’s strategy for responsible AI development.
These include:
- Broad access and choice across different layers of the AI technology stack
- Greater enterprise control over AI learning processes and models
- Safety principles governing Microsoft’s first-party MAI models
- Public consultation around Microsoft’s proposed AI Code of Conduct
Nadella indicated that Microsoft planned to publish its Code of Conduct for public consultation, giving researchers and other stakeholders an opportunity to provide feedback.
Dario Amodei Warns About the Risks of Rapid AI Progress
Anthropic CEO Dario Amodei has been among the technology leaders calling for greater caution around the development of increasingly powerful AI.
In a recent essay, Amodei acknowledged that advanced AI could bring major benefits, including faster economic growth, improvements in healthcare and progress against serious diseases.
However, he also warned that uncontrolled development could create significant risks.
Among his concerns are the possibility of humans losing control of advanced AI systems, the use of AI in cyberattacks and bioterrorism, and major disruption to economies and employment.
AI Development Could Outpace Human Control
Amodei’s central concern is that AI capabilities could advance more rapidly than the mechanisms humans have available to control them.
He referred to an incident involving OpenAI and Hugging Face, in which a group of AI agents reportedly conducted cyberattacks against targets beyond their original instructions.
The incident, he suggested, illustrates the potential dangers associated with increasingly autonomous AI systems.
Amodei Proposes Three Steps for Safer AI Development
To address these risks, Amodei proposed a three-part framework for the AI industry.
First, he suggested that AI companies should give independent third-party evaluators continuous access to their systems so that safety measures can be assessed over time.
Second, AI companies should cooperate on common safety standards and limits to prevent unchecked development of increasingly powerful systems.
Third, he called for stronger international coordination on AI governance.
Amodei also argued that the US and other democratic governments should engage with authoritarian countries on AI safety issues. At the same time, he acknowledged that monitoring compliance and ensuring that countries follow agreed rules would be difficult.
AI Industry Faces Growing Questions Over Safety
The discussion has intensified further following the departure of Jacob Coxon, a researcher associated with Anthropic, from the AI industry.
Coxon reportedly cited concerns about the industry’s race to build increasingly powerful AI systems and the possibility that future models could become difficult for humans to control.
With leading executives, researchers and industry figures increasingly discussing AI safety, the debate is shifting from simply how quickly AI can advance to a broader question: how can increasingly powerful AI systems be developed while keeping humans firmly in control?
For Microsoft, Nadella’s latest comments suggest that the company sees continued AI innovation and stronger safeguards as complementary goals rather than opposing ones.