Artificial Intelligence has become nearly impossible to avoid. It writes emails, generates images, answers questions, and is increasingly being integrated into the technology people use every day. But as AI becomes more powerful, some of the people closest to its development are sounding the alarm. Recent warnings from AI researchers and industry leaders have reignited concerns about what could happen as these systems become more autonomous, capable, and difficult to control. While predictions of an AI-driven catastrophe may sound extreme, the controversy raises an uncomfortable question: If the people building this technology are warning society about where it could lead, how seriously should we be listening?

These concerns became much harder to dismiss when they began coming directly from inside the companies developing some of the world’s most advanced AI systems. On September 8th, Anthropic researcher Jacob Coxon resigned after spending three years at both Anthropic and OpenAI. In announcing his resignation, Coxon accused the companies of racing toward self-improving artificial intelligence without adequately addressing what could happen if humans eventually lose control. He warned that the people developing these systems genuinely believe AI could threaten humanity’s survival by the end of the decade.

What made Coxon’s warning particularly alarming, however, was that he was not alone. Evam Hubinger, an alignment science lead at Anthropic whose work focuses on ensuring AI systems remain aligned with human goals, publicly supported Coxon’s concerns. Hubinger said he believes there is a greater than 10% chance that AI could kill all humans within the next decade and acknowledged that researchers still lack a clear solution for safely controlling a future superintelligent system.

The growing concern surrounding artificial intelligence has not gone unnoticed by world leaders. The United Nations has already begun pushing for greater international cooperation on AI governance, bringing governments, researchers and technology companies together to discuss how increasingly powerful systems should be monitored and regulated. The goal seems simple: establish safeguards before AI develops beyond humanity’s ability to control it effectively. However, regulating artificial intelligence may be far more complicated than simply agreeing that the technology needs guardrails.

AI development has increasingly become a geopolitical competition, particularly between the United States and China. In many ways, the race resembles the 20th-century Space Race. This time, it is not about reaching the moon first, but about which country can develop the world’s most advanced artificial intelligence. Slowing development could give researchers more time to understand AI’s potential dangers, but it could also let a geopolitical rival move ahead. The result is a dangerous dilemma: how do countries agree to slow down a technology when none of them can be certain their competitors will do the same?

Whether artificial intelligence will ever become powerful enough to pose an existential threat to humanity remains impossible to know. Predictions about where AI could be by the end of the decade are exactly that: predictions. But when researchers working directly on some of the world’s most advanced AI systems begin warning that the technology could become dangerous, their concerns should not simply be dismissed.

The challenge now is deciding what comes next. Governments may introduce regulations, companies may strengthen their safeguards, and international organizations may push for greater cooperation, but none of those efforts eliminate the competition driving AI development forward. As the United States, China, and technology companies race to build increasingly capable systems, slowing down may feel like falling behind. Yet continuing without adequate safeguards carries its own risks. We may not know exactly where artificial intelligence is taking us, but the people helping build that future are telling us to pay attention. It is time we start listening.