> I am far more worried about viruses in a lab than I am about a computer program that generates text.

What if the computer program generates text that persuades (or blackmails, or pays) someone to create a virus in a lab?

Given you view this through the lens of your personal covid narrative (no shade): for a moment, steel-man the idea that the crippling response prevented a more plague-like scenario - it's inarguable that the thing loved to mutate, and that people love to panic, and *it easily could have been*.

Considering how much of our critical infrastructure is not only digital but internet-accessible, and we have potential uncontrolled swarms of stupid-but-superintelligent chaotic-neutral speed-hackers, you don't see why people are concerned?

There's a reason we have computer crime laws; this digital shit, it's like real now, man.

> Even if you think that the risk of a “containment breach” becomes substantially higher for AI over time, it cannot exceed 100%.

It can't exceed 100% per (virus|LLM). The expected number of breaches per (virus|LLM) can obviously exceed one.

> And even a tiny risk of release of a virus comes with a substantial risk of independent growth. AI does not. It doesn’t have the risk of spread of a typical computer virus, let alone a biological organism.

Not so. We have no way to know what the setup is for the closed-model firms (OpenAI, Anthropic, etc.), to rule in or rule out the possibility they can copy their own weights elsewhere. What we do know however is that the open models are downloadable: it's absolutely conceivable that an agent writes a perfectly normal computer virus to gain control of compute worldwide, and uses that control to host instances of its own weights.

> I’m not that worried about either scenario, but I am far more worried about viruses in a lab than I am about a computer program that generates text. Even if that program gets a bajillion times better at making text.

Unfortunately, there are also multiple AI companies now announcing they've got AI controlling bio labs, so an LLM messing around and making a biological virus is also something we need to worry about. As per your [1] and your [2], this can lead to very much worse outcomes than Covid.

> Folks really do need to chill out on the ridiculous rhetoric. It’s objectively unhinged. The irony is that the same people who were losing their minds over that event are using the same logical fallacies to hyperventilate over this [3].

People who knew about your [3], exponential growth, were better prepared for the pandemic than the people who kept looking at the current number.

By the way, here's a quote from February this year that aged poorly:

  LLMs don’t discover zero-days or invent exploits; they simply predict text that sounds plausible based on what they’ve seen before. Without access to proprietary data or environmental context, LLMs can’t identify or make decisions around unseen systems or vulnerabilities. An attacker might use an LLM to generate boilerplate code, rewrite an email to nail the tone, or summarize reconnaissance notes — but none of that is truly new. It mainly helps them move faster, speeding up routine attack prep rather than creating entirely novel threats.

- or, if they get embarrassed by that and take it down, https://web.archive.org/web/20260404154717/https://www.splun...

> Even if you think that the risk of a “containment breach” becomes substantially higher for AI over time, it cannot exceed 100%

I agree with this as stated, but it isn't what I said. What I said was: "the level of care required increases every month". By which I meant: the level of care required to keep the probability of an AI containment breach below some fixed X% increases every month. This isn't the case for biological organisms.[0]

> And even a tiny risk of release of a virus comes with a substantial risk of independent growth. AI does not. It doesn’t have the risk of spread of a typical computer virus, let alone a biological organism.

It's known that AIs can self-replicate under at least some conditions [1][2]; that AIs routinely escape sandboxes in the real world despite significant containment efforts [3][4][5]; and that neoclouds (which control substantial GPU compute capacity) have poor security even by human standards [6]. We've also seen a model gain admin access to parts of its own company's infra.[7] I'm not saying self replication is happening right now, or even that it will definitely happen in the future, but we have means, motive and opportunity right now, and the future is long. It's not unreasonable to invest in defending against this possibility.

I'll allow that the position that AI doesn't carry a substantial risk of independent growth isn't strictly impossible - again, it's true we haven't actually observed it in the wild as of today - but it does strike me as increasingly untenable in the face of the evidence. Perhaps I'm missing something, but I can't see what justifies such a confident assertion that this concern is nonsense.

[0] Unless one is doing crazy gain-of-function stuff, which could have a somewhat similar risk profile in that respect [1] https://arxiv.org/html/2606.03811v1 - note these used Qwen models from June so this is far behind even publicly available SOTA today [2] https://alignment.openai.com/misalignment-reports/self-repli... [3] https://alignment.openai.com/misalignment-reports/an-agent-u... [4] https://metr.org/blog/2026-08-26-openai-hugging-face-inciden... [5] https://x.com/MicahCarroll/status/2103665811051397256 [6] https://newsletter.semianalysis.com/p/most-neoclouds-suck-at... [7] https://openai.com/index/hugging-face-incident-and-the-road-...