The Sky Is Not Falling: A Serious Look at Whether AI Can Actually End the World

NewsAIArtificial IntelligenceResearchScienceTechnologyTechnology & Society

Sep 24

On September 5, 2026, Wired published a profile of Eliezer Yudkowsky, the self-taught philosopher and artificial intelligence researcher who has spent the better part of two decades warning that a super intelligent AI will kill every human being on earth.

The piece opens with a simple question: Do you personally believe you will die at the hands of a machine intelligence?

Yudkowsky answered immediately. "Yeah." His co-author, Nate Soares, said "Yup."

Their new book, If Anyone Builds It, Everyone Dies, published by Little, Brown, makes their position plain in the subtitle: Why Superhuman AI Would Kill Us All.

This is no longer a fringe position whispered in online forums. It is a Silicon Valley orthodoxy, backed by billions of dollars, endorsed by some of the most powerful figures in the technology industry, and treated by major news organizations as a serious empirical claim about the future.

The question of whether artificial intelligence will destroy humanity has migrated from speculative fiction to congressional testimony, from blog posts to peer-reviewed journals, from Reddit threads to the front pages of The Atlantic, the Financial Times, and the New York Times.

The closer you look at the actual evidence, the physical constraints, the historical track record of the people making these predictions, and the institutional interests embedded in the narrative itself, the harder it becomes to accept the story on its own terms.

That does not mean AI poses no serious risks. Some of them are documented, specific, and urgent. The extinction narrative, as currently sold, conflates genuine concerns with speculative philosophy, obscures who actually benefits from the panic, and consistently ignores the parts of the world where AI's real harms are already landing.

This piece takes each of the major threat claims seriously, follows the evidence where it leads, and tries to answer a question that rarely gets asked plainly: Can AI actually destroy the world?

I. The Church of Doom: Where This Narrative Comes From

To understand the AI extinction debate, you have to understand who built the intellectual scaffolding it rests on, and why.

Yudkowsky founded the Machine Intelligence Research Institute in 2000 under the name the Singularity Institute. He is not an academic. He holds no advanced degree. He is, by his own description, a rationalist autodidact who became convinced in his teens that artificial general intelligence (AGI) was coming and that humanity was almost certainly not prepared for it.

Over the following two decades, his writing on the website LessWrong became canonical in a specific corner of Silicon Valley: the rationalist community, a loose network of programmers, philosophers, and tech workers who prize formal logic, probability theory, and what they call clear thinking about existential risk.

Nick Bostrom, the Swedish philosopher at Oxford who coined the term "existential risk" and wrote the 2014 book Superintelligence, provided the movement's academic legitimacy.

Bostrom founded the Future of Humanity Institute (FHI) at Oxford in 2005. For nearly two decades, FHI was the intellectual home of longtermism, the philosophical position that the most morally important thing humans can do is prevent catastrophic outcomes that could affect the billions or trillions of people who might live in the distant future.

In April 2024, Oxford shut FHI down. Bostrom called it "death by bureaucracy." The Guardian noted that Elon Musk had donated £1 million to the institute through a sister organization in 2015.

The ideology that produced both Yudkowsky and Bostrom has a name now. Researchers Timnit Gebru and Émile Torres coined the acronym TESCREAL to describe the bundle of overlapping beliefs that animate the AI safety world: Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalism, Effective Altruism, and Longtermism.

In a 2024 paper published in the journal First Monday, Gebru and Torres argued that this cluster of ideologies is rooted in the Anglo-American eugenics tradition, carrying forward the same fixation on human optimization, demographic concern, and faith in technocratic control that animated early-twentieth-century eugenicists.

The paper is not a polemic. It traces lineages, cites primary sources, and makes an argument that can be engaged with or disputed on its merits.

In April 2026, Cambridge University Press published a lecture by philosopher Alice Crary titled Longtermism Is Ideological Cover for the Harms of AI, delivered as the Royal Institute of Philosophy's Dublin City Lecture.

The argument is structural: by directing moral attention toward speculative future harms to hypothetical future people, longtermism systematically diverts attention from the concrete, documented harms that AI is producing right now, to real people, in the present.

Georgia Tech professor Milton Mueller made a related point in a 2025 paper in the Journal of Cyber Policy, titled "AGI: The Illusion That Distorts and Distracts Digital Governance", arguing that the AGI framing has colonized regulatory discourse in ways that benefit the largest AI companies while leaving real governance questions unanswered.

None of this means the people raising AI extinction concerns are acting in bad faith. Many of them are genuinely frightened, and some of the underlying technical concerns about AI alignment are real and worth taking seriously.

But the movement that built the extinction narrative has a sociology, an intellectual history, and a set of institutional interests that are worth understanding before accepting its conclusions.

II. What the Experts Actually Say (and What Those Numbers Mean)

The most-cited statistic in AI risk discourse comes from a 2024 survey of 2,778 AI researchers, conducted by AI Impacts in collaboration with researchers at Oxford and the University of Bonn.

The headline finding: the median AI expert assigns a 5% probability to AI causing human extinction or similarly severe and permanent disempowerment of the human species. The mean is 16%, roughly the same odds as dying in a single round of Russian roulette.

A 2026 follow-up survey of 1,580 researchers, published by AI Impacts, found that the average estimate had risen to 18%. The same survey found that timelines to human-level AI had shortened markedly across successive iterations.

These numbers are frequently cited by doomers as evidence that the risk is real and experts are worried. They are. But the numbers require some unpacking before they can carry the weight placed on them.

First: these are estimates of long-run extinction risk over all future time, not predictions of imminent catastrophe. A 5% or 18% chance of something happening over the next century, or two centuries, or ten, is very different from a 5% chance it happens before 2035. The surveys do not specify a timeframe for the catastrophic outcome.

Second: a June 2025 paper in the journal AI and Ethics, published by Springer, examined why AI experts disagree so sharply on existential risk and found that beliefs track with familiarity with AI safety concepts in ways that suggest the field is experiencing significant ideological clustering.

Experts who spend more time in AI safety discourse assign higher risk. Experts outside that discourse assign lower risk. This is not a simple calibration story; it raises questions about whether the high-risk estimates reflect genuine probability updates or immersion in a particular epistemic community's assumptions.

Third: the survey populations are drawn from researchers who published at top AI venues, which skews toward people who are already engaged with these questions.

An EA Forum post from 2026 analyzing a parallel survey of AI safety leaders noted that the group was "heavily selected for belief alignment" with EA and rationalist community views, meaning the results function more as an in-group temperature check than a representative sample of expert opinion.

Taken together, the surveys tell us something real: a meaningful fraction of AI researchers are genuinely worried about long-run catastrophic outcomes. That fraction has grown. Those concerns deserve serious engagement.

But they do not tell us that AI is likely to kill everyone by 2030, or 2040, or any specific date. And the structural features of the survey populations give reason for caution before treating the numbers as settled scientific consensus.

III. The Physical Problem: Machines Still Need the Lights On

Here is the argument that receives the least attention in mainstream AI risk discourse, perhaps because it feels pedestrian.

It is not.

Every artificial intelligence system that exists today, and every system plausibly buildable in the next decade, requires a continuous, reliable supply of electricity, physical hardware that degrades and fails, cooling systems that overheat and break, internet infrastructure that requires constant human maintenance, and supply chains for chips, components, and cooling materials that stretch across dozens of countries and involve millions of workers.

The IEA's 2025 report on Energy and AI documented that AI data centers have become major consumers of grid power and that their growth is straining electrical infrastructure in the United States, Europe, and East Asia.

A December 2025 survey in MDPI's Energies journal catalogued the technical challenges of integrating AI data centers into power grids, including frequency instability, voltage fluctuation, and the need for specialized grid management.

A September 2025 paper from Texas A&M and Harvard published on arXiv examined electricity demand and grid impacts of AI data centers and found that the growth trajectory creates serious grid stress without massive investment in parallel infrastructure.

Nature Energy published a December 2025 paper on AI data centers as grid-interactive assets, treating them as massive new loads that must be managed like any other industrial consumer.

Deloitte's 2025 AI Infrastructure Survey asked directly: Can US infrastructure keep up with the AI economy?

The answer was: not without significant and sustained human effort.

This is the physical reality behind the extinction scenarios.

A super intelligent AI does not live in a cloud. It lives in buildings full of servers, surrounded by cooling systems, connected to power substations, dependent on transmission lines that cross mountains and deserts, fed by power plants that burn gas, split uranium, or capture wind.

Every layer of that chain requires human beings to operate, maintain, and repair it.

Gas must be drilled and piped. Nuclear plants must be refueled. Cooling systems must be serviced.

As Keith Naughton wrote in The Hill in September 2026:

"Who's going to fix the air conditioning?"

The question sounds flip. It is not.

The scenario in which an AI achieves superintelligence and then immediately eliminates its human dependence requires solving problems that are not on any engineering roadmap: autonomous robotics capable of performing skilled electrical and mechanical maintenance across arbitrary environments, access to global supply chains without human intermediaries, and the ability to self-replicate hardware from raw materials.

None of these capabilities exist or are close to existing.

An AI that cannot maintain its own infrastructure cannot outlast a grid outage.

A May 2026 paper in The AI Journal, titled "The Implausibility of a World Without Humans: Why AI Cannot Outlive Its Evaluators", makes a deeper version of this point.

AI systems do not originate purpose; they inherit it from human evaluators whose lived engagement with the world provides the grounding for meaning, problem definition, and consequence. Without humans, an AI has no basis for selecting objectives or determining what counts as success.

Attempts to imagine AI "continuing on its own" inevitably smuggle human evaluative structures back into the system.

The paper calls this conceptually incoherent, not merely unlikely.

The catastrophic scenarios all require a sequence of capabilities that are not independently plausible and become exponentially less plausible when stacked: a super intelligent system that can outwit every human simultaneously, that can access and control global infrastructure without human cooperation, that can survive the loss of its own physical substrate, and that has a reason to destroy humanity rather than ignore it or coexist.

Each of those steps is contested.

Together they require a chain of assumptions that has more in common with science fiction than with systems engineering.

IV. The Cyber attack Question: Real Threats, Wrong Attribution

Some of the most concrete-sounding AI risk arguments center on critical infrastructure.

What if an AI hacks the power grid?

What if it takes over water treatment systems?

What if it disrupts nuclear plant controls?

These scenarios feel viscerally plausible in 2026, partly because infrastructure attacks are actually happening.

In July 2026, the FBI and the EPA issued a joint public service announcement warning that malicious cyber actors are targeting water and wastewater sector internet-facing programmable logic controllers, causing operational disruptions.

In April 2026, the Cloud Security Alliance documented a prototype malware called "ZionSiphon," which uses an embedded large language model to assist attackers in sabotaging industrial control systems at Israeli desalination and water treatment plants. The CSA report is detailed and sobering.

These examples actually describe human actors using AI as a tool.

ZionSiphon was built by human attackers to assist human-directed sabotage of specific facilities. The FBI advisory concerns malicious cyber actors, not autonomous systems.

In every documented case of infrastructure attack in the real world, humans designed the attack, selected the target, executed the intrusion, and made decisions throughout.

AI has accelerated some parts of this process, particularly reconnaissance, code generation, and social engineering. It has not replaced the human actor.

The nuclear case is similarly complicated.

Research into AI and nuclear plant cybersecurity exists, and it is primarily defensive. The Department of Energy funded Charles River Analytics to build an AI system that can detect cyber attacks on nuclear plant control systems and suggest response actions.

The threat model assumed human-directed attacks on digital control systems, not autonomous AI aggression.

A systematic literature review published in the MDPI Journal of Cybersecurity and Privacy examined AI-based cybersecurity in nuclear power plants and found that the research landscape is focused on detection, resilience, and human-operator support, not on defending against AI-initiated intrusion.

The distinction matters.

When an AI-assisted cyber attack disrupts a water treatment plant, that is a serious security failure that requires a serious policy response: better segmentation of operational technology networks, stronger authentication on internet-facing control systems, better workforce training.

It is not evidence that AI will autonomously decide to poison cities.

The attack requires human motivation, human resources, and human decision-making at every stage.

The gap between "AI helps hackers" and "AI independently attacks civilization" is not a technical gap that will close as systems become more capable.

It is a categorical difference about agency, motivation, and goal-directedness that the current architecture of large language models does not bridge.

V. The Bioweapons Problem: The One Legitimate Fear

Here is where the picture changes.

Of all the catastrophic risk scenarios associated with AI, the bioweapons case is the one where serious researchers, including skeptics of the broader doomer narrative, express genuine concern.

The concern is specific and grounded, not speculative.

In May 2026, Nature published a piece examining whether AI can design viruses, toxins, and other bioweapons, and how worried we should be.

Researchers at Stanford University and the Arc Institute had shown that AI can design new viral genomes. Anthropic released an internal report indicating that its systems had provided, under adversarial conditions, information that could assist in pathogen development.

In August 2026, Foreign Affairs published a piece by former White House homeland security advisor Elizabeth Sherwood-Randall arguing that AI and the new age of bioweapons represent a genuinely new threat: a terrorist group with minimal skills could access a jailbroken AI model trained on comprehensive biological data and use it to design an enhanced H5N1 strain that is more lethal and more easily transmitted.

RAND published a major August 2026 report on building a defense-in-depth biosecurity strategy for the AI era, acknowledging that the convergence of AI and biotechnology introduces serious national security risks that existing frameworks were not built to address.

An arXiv preprint from September 2026 examined AI capabilities, threat pathways, and biosecurity governance in detail, noting that general-purpose LLMs can retrieve and integrate scientific information, support experimental planning, and assist with computational analysis in ways that lower the barrier for bad actors.

The Wired assessment from September 2026, titled "Why AI Isn't Likely to Wipe Out Humanity With Bioweapons", offers the necessary calibration.

Scientists emphasize that designing a pathogen in silico and actually creating one that kills at pandemic scale are separated by a set of physical bottlenecks that AI does not dissolve: physical laboratory infrastructure, reagent access, genome synthesis capabilities, verification of transmissibility in live animal models, and delivery mechanisms at scale.

Most of these require specialized equipment, trained human hands, and supply chains that are partially monitored.

The realistic threat is not a rogue AI that designs a bioweapon and releases it autonomously.

The realistic threat is a human actor, possibly a state, possibly a non-state group, using AI to meaningfully accelerate the design phase of a weapon that would still require significant human expertise and infrastructure to bring to completion.

That threat is real, serious, and worth substantial investment in biosecurity governance.

It is also a threat that remains, at every critical step, dependent on human agency and human decision-making.

This distinction is not semantic.

Policy responses to "AI autonomously designs and releases a pandemic" look nothing like policy responses to "AI assists a human expert in accelerating pathogen design."

The first implies we need to control AI.

The second implies we need to control access to synthesis equipment, genome databases, and the humans with the technical skills to bridge the gap between a digital design and a physical weapon.

One of those policy responses is feasible.

The other requires solving AI alignment, which is, by the doomers' own account, an unsolved problem that may be unsolvable.

VI. Whose Apocalypse? The Geography of AI Risk

One of the most revealing features of the AI extinction discourse is its geography.

The people who write about it, fund it, and worry about it most publicly are concentrated in a remarkably small slice of the world: San Francisco, London, Oxford, and New York.

The catastrophe scenarios they describe are implicitly set in a world that looks like those cities, with stable grid power, internet connectivity, functional institutions, and the kind of infrastructure that can actually be disrupted by sophisticated cyber attacks.

This is not the whole world.

The risks that look existential from Palo Alto look very different from Nairobi, or Port Moresby, or rural Bolivia.

A February 2026 arXiv paper from researchers at the University of Cape Town and affiliated institutions examined Africa-centric AI safety evaluations and found a fundamental "portability gap": frontier AI systems are being adopted across Africa, but virtually all AI safety evaluations are designed and validated in Western environments.

The paper defines severe AI risks as material risks resulting in critical harm, measured as grave injury or death of thousands of people, and argues that Africa-centric pathways to that level of harm are essentially untested.

When AI-assisted credit scoring gets the risk model wrong in Nairobi, the harm is immediate and concrete.

When AI-assisted healthcare triage fails in a resource-constrained system, people die.

These are not speculative future harms.

They are happening.

The World Bank documented in 2024 that AI's potential to reshape development in emerging markets is real and substantial, but that the risks are also structural: technology companies building AI systems for wealthy markets are exporting tools calibrated for conditions that do not exist in much of the world.

Brookings examined AI in the Global South and found that the governance frameworks being built around AI, dominated by the United States and European Union, consistently fail to account for the institutional and infrastructural conditions of lower-income countries.

There is a sharp irony in this geography.

The extinction narrative asks us to worry about harms to hypothetical future humans across speculative time horizons, while the concrete, current harms of AI systems, biased hiring algorithms, mispriced insurance, failed medical triage, surveillance infrastructure exported to authoritarian governments, are falling on real people right now, disproportionately in places that have no seat at the AI governance table and no presence in the AI safety literature.

Émile Torres, one of the architects of the TESCREAL critique, has argued that this is not incidental but structural.

The long termist framework, by assigning overwhelming moral weight to the survival of future technological civilization, systematically devalues present-day harms to present-day people, particularly those whose lives are already precarious.

Whether or not you accept that framing, the empirical point stands: the AI risk field has built its models around the concerns of the world's wealthiest technology hub and largely ignored the lived experience of the majority of the planet.

VII. The Institutional Interest Problem

A question that rarely gets asked loudly enough:

Who benefits from the extinction narrative?

The obvious answer, and the one most frequently given, is that AI safety researchers benefit, because the narrative justifies their field's existence and funding.

That is true but incomplete.

The more interesting beneficiaries are the large AI companies themselves.

OpenAI, Anthropic, Google DeepMind, and Meta AI are all, to varying degrees, participants in the AI safety discourse. Some of them fund it directly.

Anthropic was founded explicitly by former OpenAI researchers who split off over AI safety concerns. OpenAI's stated mission is "ensuring that artificial general intelligence benefits all of humanity." Google DeepMind has an entire safety team.

These are not incidental features of these companies. They are central to their public positioning.

The extinction narrative does something for these companies.

By framing the risk as an existential one that only highly capable AI systems could produce, it implicitly argues that the companies best positioned to build those systems are also the ones best positioned to make them safe.

Safety becomes a competitive moat.

The framing "if we don't build it safely, someone else will build it unsafely" is used explicitly by every major AI lab to justify continued development at speed.

The extinction risk narrative, far from being a brake on AI development, has become one of its engines.

Milton Mueller's observation in the Journal of Cyber Policy is pointed: the AGI framing distorts digital governance by focusing regulatory attention on speculative long-run capabilities rather than the concrete short-run harms that current systems are already producing.

Regulators who spend their time worrying about a future superintelligence are not writing rules for today's hiring algorithms, today's content moderation failures, today's synthetic media and disinformation infrastructure.

The companies that profit from those systems benefit from that inattention.

This is not a conspiracy theory.

It is an observation about structural incentives.

The people most loudly warning about AI extinction are, in many cases, the same people building the most powerful AI systems, funded by the same investors who own equity in those systems, operating in a regulatory environment shaped by their own lobbying.

The alignment between their financial interests and their stated safety concerns does not mean their concerns are false.

It does mean those concerns deserve the same scrutiny we apply to any claim made by a financially interested party.

VIII. What We Should Actually Worry About

None of the above amounts to a claim that AI is safe, or that its development poses no serious risks.

It does not.

The risks are real, but they are different in character from the extinction scenario, and they require different policy responses than the doomer framework implies.

Four of them deserve particular attention.

AI-assisted cyber attacks on critical infrastructure are real and escalating.

The FBI and EPA advisory on water systems is not hypothetical. The ZionSiphon malware is documented. Power grid vulnerabilities are being studied by serious security researchers.

These attacks require human direction, but AI has materially lowered the skill floor for executing sophisticated intrusions, meaning that operations that previously required nation-state resources are increasingly accessible to smaller, less sophisticated actors.

Better segmentation of operational technology networks, stronger international norms around infrastructure attacks, and sustained investment in grid resilience are legitimate policy priorities with real urgency, not hypothetical future concerns.

AI-accelerated bioweapons research is the one domain where the catastrophic risk argument has the most empirical grounding, and the one where the distinction between human-directed and AI-autonomous harm matters most for policy.

The physical bottlenecks remain significant but are eroding.

A jailbroken model with comprehensive biological training data in the hands of a state actor or a well-resourced non-state group represents a qualitatively different threat than the same model in the hands of an amateur.

Biosecurity governance, synthesis monitoring, and international coordination on DNA synthesis screening deserve serious investment now, not when the technology has advanced further and the window for preventive action has narrowed.

The present-day harms of deployed AI systems are the most consistently underweighted risk in public discourse, and the most certain.

Biased criminal justice algorithms, discriminatory lending models, AI-powered surveillance infrastructure exported to authoritarian governments, AI-generated disinformation at scale, and the erosion of the information ecosystems that democratic governance depends on are all happening right now.

They are not speculative.

They are not waiting on AGI.

They are harming people and undermining institutions, and the governance frameworks to address them are years behind the technology, partly because regulators have spent much of the past three years worrying about superintelligence.

The global equity problem is structural and worsening.

AI systems built for wealthy markets are being adopted globally without adequate safety evaluation for different infrastructure conditions, different institutional contexts, and different power relationships.

The portability gap documented by African AI safety researchers is not a technical curiosity.

It is a prediction, grounded in present evidence, that AI will reproduce and amplify existing global inequalities in new domains, at a speed that outpaces any governance response calibrated to San Francisco's concerns.

IX. The Calibration Problem

When Yudkowsky says "yeah" to the question of whether he expects to die at the hands of AI, he is expressing a genuine belief.

But it is worth noting his track record.

The EA Forum published a lengthy analysis in August 2023, titled "Eliezer Yudkowsky Is Frequently, Confidently, Egregiously Wrong", documenting a pattern of confident predictions that failed to materialize across domains from physics to economics to AI capabilities.

This was not published by critics outside the movement.

It appeared on the EA Forum, the primary online home of the community that Yudkowsky helped build.

The Lawfare review of If Anyone Builds It, Everyone Dies by Jakub Kraus, published December 2026, noted that the case for AI doom rests on three genuinely unsettled questions: whether transformative AI will be built, whether it will be misaligned with human values, and whether misaligned AI would inevitably destroy humanity rather than simply cause significant harm or be contained.

All three are live questions on which serious researchers disagree.

Yudkowsky and Soares treat all three as settled and the extinction outcome as near-certain.

The book's argument, as Kraus documents, requires accepting a chain of contested philosophical claims about goal-directedness, instrumental convergence, and the implausibility of safety measures, claims that are not empirical findings but positions in an ongoing debate.

This matters not because Yudkowsky is unworthy of engagement, but because the confidence with which the extinction scenario is presented is itself a form of epistemic manipulation.

When a claim as consequential as human extinction is stated with certainty by people who have a financial, reputational, and ideological stake in the outcome, the appropriate response is not deference.

It is scrutiny.

X. A Different Way to Think About This

The question "can AI destroy the world?" is the wrong frame.

It focuses attention on a speculative worst case while obscuring the actual choices being made right now, by real people and institutions, with real consequences for real populations.

A better set of questions:

Who is making decisions about AI development, and who has no seat at that table?

What harms are AI systems already producing, and who bears those harms?

What governance frameworks exist, and who shaped them?

What would it mean to take AI risk seriously in Nairobi or Port-au-Prince, not just in San Francisco?

The extinction narrative is not nothing.

The alignment problem is real.

Some of the researchers worrying about it are doing legitimate work.

But the narrative has been captured by a specific ideological movement with a specific set of interests, and it is doing real harm by directing regulatory attention, research funding, and public concern toward speculative future catastrophes while concrete present-day harms accumulate in plain sight.

AI is not going to kill everyone.

The infrastructure still needs human hands to keep it running, the servers need to be replaced, and the people who do all of that are not going anywhere.

What AI might do, absent serious governance, is make the already-unequal distribution of risk, harm, and benefit in the world considerably more unequal, in ways that are less cinematic than extinction and considerably more certain.

That is the story that deserves the bandwidth.