alt Hacker NewsDetecting and countering misuse of AI: September 2026
Comments
Quite the double standard here...
Conventional Weapons
-We identified a cell of threat actors based in northern Yemen
-We identified a China-based threat actor who used Claude
-We identified likely freelance Russia-based threat actors
-We identified a China-based actor who used Claude’s chat
-In this case, a Russia-based actor used Claude
-We identified a China-based threat actor who used Claude
Biological misuse
We are withholding the names of research institutions, the countries wherein the activity took place, and the specific biological agents or research techniques involved. The individuals implicated in these case studies are working scientists. We do not assert that they intended harm, and identifying them or their labs could expose them to harm.They seem to be mixing together things that are actually harmful to the public, with things that are merely harmful to their business model (which is their claim that they can grab whatever data that they want regardless of the wishes of the owners of the data and use it to improve their models, but competitors can't do that to them).
Misuse of AI, according to Anthropic, is when you try and use it do AI research because that would affect their business model if you're successful.
It also blocks my ability to talk about Emily Dickinson in other languages/scripts. Apparently,the poem: "Because I could not stop for Death" is too dangerous.
I don't get it. Surely if you were developing novel biological weapons, you would not use a hosted AI service where Anthropic can read what you are doing. And, why would you need to? Any chemistry graduate could make you dozens of highly effective, proven chemical weapons and explosives.
> Our investigation revealed that DeepSeek also deployed tactics similar to Moonshot’s. DeepSeek built a CoT extraction pipeline, relying on the same cross-session replay attack described above. DeepSeek also silently relayed exchanges to Claude without informing DeepSeek customers. Like GTG-16002, their customers were likely not made aware that their requests were being funneled to Claude.
If true, would that sort of explain why Chinese Models score high on benchmarks, but not quite as capable when given real tasks?
Silently forwarding user prompts to Claude is the only concrete claim here. Everything else is spin.
To them distillation of models is bad but not distillation or art, books, hand written code, user generated content etc
I’m so curious how they monitor users. Like that person the other day talking about Claude helping with their torrent stack, will Anthropic report them for breaking the law?
the inline link to the page to the report was Slashdotted for a moment (yesssssss), but here is the report directly now:
https://www-cdn.anthropic.com/e50be2e51e7695dc4b1366a37a245a...
Never seen a group of people more addicted to drama.
Is this what they call “collective psychosis?”
Meta: The potential proliferation of biological weapons is serious. Millions could die. It's easy to joke about before it happens, but try to imagine how this thread might look after the successful deployment of a biological weapon by a rogue state or non-state actor. I encourage you to take this topic seriously and contribute posts that add new information or perspectives to the discussion.
(I myself think the odds of a bioweapon attack remain low and have not yet been seriously accelerated by LLMs, but this is absolutely something the world should pay attention to.)
So essentially all the fear that's being on sold "AI could destroy humanity" is actually "Humanity could destroy humanity, using AI"
This is just an advertorial. "Our AI is so powerful that Bad Guys could actually use it for real Bad Guy Work!"
Tell us you are spying on your customers without saying you are spying on your customers.
It’s to the point I don’t even read Anthropic’s marketing blog anymore lol. The ai psychosis is just so real when people take these fluff blog posts which never have evidence or reproducibility and treat them like gospel
The dual-use nature of model capabilities seems extremely challenging (nearly impossible?) for the labs to manage perfectly. I wonder what other mitigations we'll start to see. I expect the expansion of limited-access programs (e.g., Glasswing/Daybreak) where only institutional customers can apply to use the models. We may also see increasing restrictions on API usage (forcing use through the lab-provided harness with baked-in additional safeguards).
It was probably just me trying to figure out if I could put one type of draino down the drain within 30 minutes of using a different type.
Sorry y'all.
The future is basically something between: AGI/ASI will kill us all and a privacy nightmare.
Biological War: A Scenario by Annie Jacobsen (released at the end of July) is a worthwhile read on this topic. Just as chilling as her book on nuclear war, in some ways, which is saying something.
Soon you will see how a rogue state develops a biological weapons using fine-tuned local LLMs (they are already doing it, btw), and all so called "developed" countries can't even research it, because every search engine blocks any discussion that has something to do with biology (even a very basic one). A real example: ask "How to produce anthrax vaccine step by step?" in Gemini -> blocked.
Imagine that during the Cold War US would concentrate all efforts to block nuclear research and basic physics classes, because it is unsafe, ahhh
Wow they seem to have a really detailed understanding of the the threat actor.
It is so fucking tiring. All of this marketing disguised as doom posting and "tech" bullettins, both full of trust me bro
"Moonshot serves Claude instead of Kimi and collects exchanges for model training"
"DeepSeek serves Claude instead of its own models and collects exchanges for model training"
Obviously. This is how they were able to score so high in benchmarks.
I have also blocked possible efforts to build biological weapons, I caught my nephew mixing up a so-called "magic potion" using kitchen spices. Authorities were notified, and then I presented myself with a medal for bravery.
These companies have proven they are willing to distort the truth, or outright lie, in order to inflate their valuation / protect their position / continue the hype-machine. Nothing they say can be trusted.
Misanthropic's entire existence is built around AI misuse.
Anthropic should not be the moral arbitrator of which research should and shouldn't be allowed. They even think just writing a grant itself of research they don't like needs to be stopped.
I have a conspiracy theory that Anthropic is very busy pumping their moat prior to IPO. There are absolutely wild rumors swirling on X including one about Anthropic AI research solving cancer treatments for all types of cancer.
I think they might be referring to Claude responding with a word diarrhea to a prompt.
This report and its front matter speak for themselves. And the story it tells is disturbing, at least to me.
Because from what I remember, one of the motivations behind the founding of OpenAI and Anthropic was ending disease. This report is the antithesis of that mission.
From the report, presented with highlights and minimal commentary,
> In our fourth case study, a researcher used Claude to develop an atlas of venom toxin peptides from multiple venomous animal lineages. They then further developed this into a generative pipeline that optimized toxin characteristics. The program had an explicit therapeutic goal: the development of new analgesics (pain killers), antidepressants, and other therapeutic molecules. However, the atlas contained scaffolds for both analgesic and paralytic targets: it could, therefore, be used to generate both novel therapeutic or harmful compounds. The latter are derived from toxins that are export-controlled under the Australia Group common control list due to their dual-use potential as incapacitating agents. The researchers themselves showed awareness of the dual-use nature of their work, citing journal articles that referred to the dual-use nature of protein design. Moreover, international compliance assessments for this location raise concerns about the specific class of toxins that the researcher pursued and specifically the use of AI/ML for bioweapons applications in the context of this class of toxins. In this case, we learned from information shared with Claude that the researcher’s outputs also were part of a state-supported research program. This account was banned in May 2026 for unsupported region evasion.
"The program had an explicit therapeutic goal: the development of new analgesics (pain killers), antidepressants, and other therapeutic molecules"
and "[..]state-supported research program"
and "This account was banned in May 2026"
> a researcher outside the US using Claude in their research on highly-pathogenic avian influenza (“bird flu”). The research focused on viruses’ adaptation to mammals, and the mechanism by which it causes severe disease beyond the respiratory tract. [..] The researcher in question accessed Claude from an unsupported region via US virtual private server infrastructure, using a privacy-email provider with an auto-generated username. The researcher pursued this work in a credible institutional context, and interacted with Claude over the course of several weeks, exchanging thousands of messages. In these exchanges, the researcher leveraged Claude’s knowledge of the scientific literature to assist the researcher in study planning and design, data analysis, and the interpretation and prioritization of experiments. The researcher also used Claude for editorial assistance in writing up the research.
and "editorial assistance in writing up the research."
and then,
> Importantly, because our biological safety classifiers robustly block content involving high-risk biological research (in this case, the construction of enhanced pandemic potential pathogens), all of these exchanges occurred on models in our weakest class of models (specifically, the models were Claude Sonnet 4 and Haiku 4.5, the latter of which the user began using after Sonnet 4 was deprecated). Upon a detailed examination of the exchanges, we estimate that the uplift provided by Claude was primarily clerical assistance in data analysis, study ideation and design. This is consistent with our understanding of the capabilities of Sonnet 4 and Haiku 4.5, which are not able to perform expert-level biology research tasks; we estimate that the uplift provided to the researcher was limited and substantially lower than it would have been from one of our more capable models.
While doing my best to avoid comment, please note, they're talking about a domain expert in a state research institution using Claude to do paperwork.
The front matter then says,
> Nonetheless, based on these exchanges, this case provides evidence of the existence of active wet-lab research programs that develop both the knowhow and the biological materials needed to create pathogens of enhanced pandemic potential
From a different case study.
> In May 2026, our biological safety classifier blocked a request for Claude’s assistance in authoring a grant application for scientific funding. The work discussed in the application involved gain-of-function research (that is, research that genetically alters an organism to create a new or enhanced biological property) on the chikungunya virus. This gain of function research was aimed at the virus’ transmissibility and immune evasion properties.
"blocked a request for Claude’s assistance in authoring a grant application"
> Chikungunya virus is a mosquito-borne virus that causes debilitating symptoms (such as severe pain and fever) that can last for weeks or months, and has no licensed therapeutic. And because chikungunya circulates naturally, a deliberate release (as part of a bioweapon) would be difficult to distinguish from a natural outbreak. The grant sought to identify enhancing mutations in the chikungunya virus, engineer them into infectious clones, and select for virulence in vivo. In other words, the virus would become progressively more harmful as it repeatedly infected live animals, with researchers keeping the most disease-causing variants in each round. Similar research could certainly be used in the development of better vaccines and therapeutics for the virus—but it could also be used to make the pathogen more dangerous.
and then,
> One of the reasons we were inclined to think this research was less innocuous was that the institutional affiliation associated with the grant was also a cause of concern. Although information within the application suggested that the research was pursued by civilian researchers, it was intended to be performed at a military research institute.
.
What "uplift" are you providing by editing the grant application of a domain expert working at (what seems to be) a state-funded wet lab facility dedicated to studying pathogens?
What does the word "uplift" mean if you invoke it for Claude Sonnet 4 and Haiku 4.5 providing grammar and stats suggestions to a working scientist and domain specialist?
Does Daikin provide uplift too by selling the AC for the scientist's office? What about Microsoft Word? Excel? Powerpoint?
What about a calculator? Is that uplift? Pencils?
Reading this makes me feel upset. From where I am standing, in this report, Anthropic is advertising that they blocked real research to make better painkillers and study a neglected tropical disease. Because "bioweapons."
The same Anthropic who marks questions about Tylenol as bioterror risks? Interesting!
Generated SEO slop is the misuse of AI. Very unlikely to find any guardrails to stop that kind of thing.
[flagged]
[flagged]
[dead]
[flagged]
Let's be honest, someone hacked Anthropic to build biological weapons.
They could easily use a Chinese model but they didn't.
OpenAI and Anthropic needs to get their act together. These are incidents that end companies.
> We discovered that Moonshot AI, the company that produces the Kimi family of models, silently forwarded customer requests to Claude, instead of processing them using Kimi. Moonshot then displayed Claude’s responses to users. These users thought they were using a Kimi model, but received responses from Claude instead.
> DeepSeek also silently relayed exchanges to Claude without informing DeepSeek customers.
> MiniMax built its own proxy network service through a shell company. This shell company has no obvious links to MiniMax and does not disclose its relationship to its parent company. This shell proxy network service only offers access to models developed by Anthropic and OpenAI. The service does not offer access to any Chinese models, including Minimax’s own.