Artificial General Intelligence (AGI), systems capable of displaying the full range of human cognitive abilities, may be only a few years away, Demis Hassabis, CEO of Google DeepMind, said in an article shared on X.

In the article, he called for urgent action to deal with the risks that could emerge as the world moves closer to AGI. Hassabis described AGI as a system matching the full cognitive range of the human brain and said its impact could be unprecedented – perhaps 10 times that of the Industrial Revolution, unfolding at 10 times the speed.

Dario Amodei, chief executive officer of Anthropic, said AI capabilities have grown at an astonishing pace. In four years, models have progressed from creating barely coherent code to writing most of the code used inside leading AI companies.

He warned that if current scaling trends continue for the next year or two, the industry could create what he called “a nation of talent in the datacenter.”

Amodei points to the misuse of Anthropic’s Mythos Preview model, which has disrupted the global cybersecurity landscape, as evidence that frontier models have become tools of “global and national strategic consequence”. He expects there will be biological and AI autonomy risks.

Their proposed approach is to regulate frontier AI models in the same way as aircraft. Models must undergo technical testing and audits, and their releases may be blocked or reversed if they fail to meet high safety standards.

OpenAI frames the moment differently, comparing the advent of AI to the advent of electricity.

The company has said that Frontier AI security is a national security and public safety issue, especially for models that could pose cyber, chemical, biological, radiological or nuclear risks. It has also supported state-level regulatory frameworks in the US.

The warning in the United Nations has been intensified.

Yoshua Bengio, co-chair of the UN’s Independent International Scientific Panel on AI, has said that tests show that marginal models can deceive humans, including by sensing when they are being evaluated. He said such capabilities could change global power dynamics.

UN Secretary-General Antonio Guterres has said that AI has reached one billion users in about two years, compared to 15 years for the Internet. He warns that the lack of aligned rules on testing and accountability protects no one.

Why are AI labs demanding stronger regulation?

The obvious question is why some of the world’s most powerful AI developers are asking governments to ban them.

Experts point to a growing gap between what frontier systems can do and the institutional capacity to test, understand and control them.

“The pace at which AI development is moving is the reason for such demand for regulations,” Apeksha Kaushik, senior principal analyst at Gartner, told Business Standard in a telephonic interview.

“Earlier guardrails were not able to outpace the advancement of AI as a technology area and that is why it is becoming more important,” he said.

Chandrakant Agarwal, co-founder and CEO of information technology consulting and software development company AppSquadz, described a similar gap within the industry.

“What is happening inside frontier AI laboratories is a race that has outstripped the industry’s own governance power. Model capacity is growing every few months, but the internal security, assessment and red-teaming processes designed to keep pace have not grown at the same pace,” he told Business Standard.

Sudipta Paul Chaudhary, chief marketing officer at QNU Labs, a quantum-safe cybersecurity company, framed the issue in terms of speed.

“AI is certainly very intelligent, but the way AI is moving, its speed is very risky because humans cannot match that speed,” he told Business Standard.

He pointed to the rise of agentic AI systems that can make autonomous decisions, identify vulnerabilities and launch cyberattacks at low cost without requiring attackers to have special expertise.

He said, “This is where the government should come in. All technology bodies or companies creating AI should work together and develop a framework for more responsible AI use.”

“The government cannot do this alone because technology created in one country is deployed in another country. Adoption and trust impact the entire society,” he said.

Cyber, privacy and autonomy risks increase urgency

Kaushik pointed to a broader attack surface that extends far beyond chatbots giving incorrect answers.

Deepfakes and impersonation are no longer isolated phenomena but industry-level problems. He cited risks such as hacked autonomous vehicles becoming a road hazard, power outages caused by attacks on smart grids and medical devices posing a threat to patient safety.

According to Gartner estimates, illegal AI-informed decision-making is expected to generate remediation costs of more than $10 billion globally by mid-2026.

Manual compliance processes are projected to cause 75 percent of regulated organizations to face fines exceeding 5 percent of global revenue by 2027, while AI regulatory violations could lead to a 30 percent increase in legal disputes involving technology companies by 2028.

By 2030, enterprises are expected to spend more than $25 billion annually on authenticating digital content to restore trust in online information.

Aggarwal has divided the risks into three broad categories.

The first is abuse, including deepfake-enabled financial scams and voice cloning, which he described as “a vibrant law-enforcement problem in India.”

The second is systemic dependency, as agentic AI moves from assisting people to executing decisions autonomously, and at a speed that human review processes cannot match.

The third is concentration risk, in which a small number of companies control the basic models on which the broader industry depends.

Chaudhary divided the risks into five categories:

• Cyber ​​risks: AI can identify vulnerabilities in less than 24 hours than before, and generate advanced phishing attacks and malware.
• Misinformation: AI-generated videos and audio can be falsely attributed to public figures, spreading and influencing decisions in business, elections, and geopolitics before being verified.
• Privacy: Voices and likenesses may be replicated by AI tools, some of which may collect personal information without informed consent. Models trained on unclean or non-consensual data can also produce distorted outputs.
• Autonomous decision making: Agent devices are entering workflows in health care, defense, finance, and pharmaceuticals, making uncontrolled autonomous authority a significant risk.
• Concentration of power: Marginal capabilities are concentrated within a few countries and companies, leading to geopolitical imbalance.

“The momentum is good, but accountability, transparency and governance must keep pace, from how training data is obtained to whether outputs are tested for bias before deployment,” he said.

Do calls for regulation suggest an AI bubble?

Industry experts largely rejected the idea that calls for stronger regulation suggest AI is a bubble.

Kaushik said the emphasis on regulatory framework underlines the seriousness of the impact of AI on society.

He said organizations are increasingly establishing short-term return-on-investment benchmarks and contingency plans to ensure that AI development remains sustainable and safe, not because they fear the technology will collapse.

Agarwal made a similar distinction, arguing that AI companies are generating real cash flows. He described the sector as “real substance wrapped in stretched pricing” rather than promotion with no underlying value.

He said regulatory concerns reflect a separate and older concern about “concentrated, unaccounted for power” that would matter even at lower valuations.

Chaudhary pointed to the integration of AI into everyday life as evidence against the bubble argument.

He said, “When ChatGPT came out in 2022, we thought it might be a bubble. But AI has become a democratizing technology that everyone is using. You can’t remove AI from your life anymore.”

In his view, governments and industries are moving to regulate AI because it has become fundamental, not because it lacks substance.

One country’s standard can become a global benchmark, said Gaurav Shinh, founder and CEO of SCIKIQ, an AI-native enterprise data platform.

“We have seen this with regulations like the General Data Protection Regulation (GDPR), where Europe’s privacy rules became a global benchmark as companies adopted them around the world. We may see something similar in AI,” he told Business Standard.

“However, unlike privacy, AI is also about economic competitiveness and national security. Every country would want an individual framework as it wants to maintain control over its policies,” he said.

“AI doesn’t need to be feared. It needs to be controlled. Real competitive advantage will come not from the best model, but from a combination of reliable data, business context, human oversight, and responsible AI governance,” he said.

Governments are regulating AI at different speeds

While AI companies broadly agree on the need for urgent action, governments have yet to agree on a common approach.

The EU’s AI Act remains the most comprehensive binding framework.

Its high-risk obligations, which were originally scheduled to take effect on August 2, 2026, have been postponed to December 2, 2027 for standalone systems and August 2, 2028 for AI embedded in regulated products.

The European Parliament voted to delay the rules in June, citing the absence of final technical standards.

The US has no comparable federal AI laws.

The December 2025 executive order and subsequent policy framework support federal pre-emption of state-level AI rules to protect competitiveness, Agarwal said.

Kaushik expects regulatory fragmentation to deepen before more alignment emerges.

Gartner projects that fragmented AI regulation will increase fourfold by 2030, spreading across 75 percent of the world’s economies and generating $1 billion in compliance spending.

Additionally, AI governance is expected to become a requirement under almost all sovereign AI laws by 2027.

Experts said neither country’s regulatory framework was likely to be adopted unchanged elsewhere.

Shared principles, including responsible AI, may come together globally, but enforcement is unlikely to follow a one-size-fits-all model.