AGI and the Dangers of Autonomous Weapons

©Prof Archie D’Souza

In 2018, Stephen Hawking posed a haunting question in his final book: “Will Artificial Intelligence outsmart us?” He cautioned that the advent of superintelligent AI could be the best or the worst thing ever to happen to humanity. Today, I am departing from my usual posts on AI applications in supply chains and project management to address a far deeper concern that keeps me up at night.

I have long argued that technology itself doesn't inherently destroy jobs—and AI won't either. Historically, however, technological leaps have frequently degraded our environment and exponentially accelerated the global arms race. On paper, AI gives us the tools to eradicate hunger, cure diseases, and optimize resource distribution. Yet, the greed of the few who control Earth's resources and sell arms to the highest bidder threatens to derail that potential.

Hawking highlighted a terrifying trajectory: "World militaries are considering starting an arms race in autonomous weapon systems that can choose and eliminate their own targets." Imagine autonomous systems carrying out catastrophic attacks like 9/11, or algorithmically prolonging devastating conflicts like those in Palestine and Lebanon—all while global governing bodies remain paralyzed in debate. Proponents of autonomous weaponry routinely ignore the most vital question: What is the endpoint of this arms race? Do we really want cheap, AI-driven lethal systems to become the digital Kalashnikovs of tomorrow, traded freely on the black market to criminals and terrorists?

This brings us to an uncomfortable geopolitical reality. When members of the UN Security Council are actively engaged in aggressive conflicts, expanding territorial threats, or maintaining global military footholds, can we realistically expect a treaty to save us? When warmongers dictate global policy, true peace feels out of reach.

Yet, acknowledging these dangers isn't about surrendering to despair. It is a call to hold technology creators and political leaders accountable before these autonomous systems become irreversible features of modern warfare.

What We Must Do

We cannot afford to treat autonomous weapons as an inevitable evolution of technology. While the geopolitical landscape is complex, public awareness and ethical regulation remain our strongest levers.

If we want to ensure AI serves humanity rather than destroys it, we must take action:

• Support International Frameworks: Advocate for and support organizations pushing for legally binding treaties that mandate meaningful human control over lethal force.
• Demand Ethical Tech Governance: If you work in tech or AI development, push for strict ethical standards and refuse to participate in the unmitigated militarization of autonomous systems.
• Elevate the Conversation: Share these discussions within your professional network. Shift the narrative from mere technological fascination to moral accountability.

Do we allow algorithms to decide who lives and dies on the battlefield, or do we demand a future where human empathy governs innovation? The choice—and the responsibility—belongs to us.