logoalt Hacker News

My minute-by-minute response to the LiteLLM malware attack

176 pointsby Fibonartoday at 3:48 PM82 commentsview on HN

Related: Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised - https://news.ycombinator.com/item?id=47501426 (483 comments)


Comments

Fibonartoday at 3:48 PM

Callum here, I was the developer that first discovered and reported the litellm vulnerability on Tuesday. I’m sharing the transcript of what it was like figuring out what was going on in real time, unedited with only minor redactions.

I didn’t need to recount my thought process after the fact. It’s the very same ones I wrote down to help Claude figure out what was happening.

I’m an ML engineer by trade, so having Claude walk me through exactly who to contact and a step by step guide of time-critical actions felt like a game-changer for non-security researchers.

I'm curious whether the security community thinks more non-specialists finding and reporting vulnerabilities like this is a net positive or a headache?

show 8 replies
simonwtoday at 4:27 PM

First time I've seen my https://github.com/simonw/claude-code-transcripts tool used to construct data that's embedded in a blog post, that's a neat way to use it. I usually share them as HTML pages in Gists instead, e.g. whttps://gisthost.github.io/?effbdc564939b88fe5c6299387e217da...

show 2 replies
felixagentaitoday at 7:19 PM

The dependency cooldown approach mentioned upthread is underrated. Most teams I've seen adopt lockfiles and pinning but still auto-merge Dependabot PRs without any delay window. The irony is that the tooling meant to keep you secure (auto-updating) is exactly what widens the blast radius of a compromised package.

The 46-minute window here is telling. If your CI/CD pipeline happens to run during that window, you're exposed. A simple policy of "no package updates within 24h of release" would have completely avoided this, and it costs nothing to implement.

show 1 reply
cedwstoday at 4:13 PM

GitHub, npm, PyPi, and other package registries should consider exposing a firehose to allow people to do realtime security analysis of events. There are definitely scanners that would have caught this attack immediately, they just need a way to be informed of updates.

show 3 replies
n1tro_labtoday at 7:18 PM

Most developers think pip install just puts files on disk and execution happens at import. But .pth files run on every Python startup, no import needed. It's not a one-time install hook like npm postinstall. It's persistent.

Shanktoday at 5:25 PM

Probably one of the best things about AI/LLMs is the democratization of reverse engineering and analysis of payloads like this. It’s a very esoteric skill to learn by hand and not very immediately rewarding out of intellectual curiosity most times. You can definitely get pointed in the right direction easily, now, though!

show 2 replies
cdcartertoday at 5:41 PM

If it weren't for the 11k process fork bomb, I wonder how much longer it would have taken for folks to notice and cut this off.

show 1 reply
agentictrustkittoday at 7:14 PM

Let me share two things if I can... 1) its genuinely useful that a comptent generalist can do first-pass incident response with AI's help now, and 2) the process overhead that keeps the ecosystem healthy does still matter. The failure mode isn't "non-experts report bugs," its "non-experts report in a way that makes triage impossible."

A pattern that worked with for us is treating package supply-chain events as a governance problem as much as a technical one--short, pre-written policy playbook (who gets paged, what evidence to collect, what to quarantine...etc), plus an explicit decision record for "what did we do and why." Even a lightweight template prevents panic driven actions like ad-hoc "just reinstall everything."

On the flip side, waiting N days before adopting new versions helps, but it's a brittle for agent systems becasue they tend to pull dependenceies dynamically and often run unattended. The more robust control is: pin + allowlist, with an internal "permission to upgrade" gate where upgrades to execution-critical deps require a person to sign off (or at least a CI check that includes provenance(sig) verification and a diff or new files). Its boring, but it turns "Oops, compromised wheel" into a contained event rather than an unbounded blast radius.

rpodrazatoday at 6:34 PM

At this point I'd highly recommend everyone to think twice before introducing any dependencies especially from untrusted sources. If you have to interact with many APIs maybe use a proxy instead, or roll your own.

anlkatoday at 6:33 PM

As I recall, the initial vulnerability in Trivy was introduced by some clawbot. The article author has 5 Claude instances running.

Maybe the author correctly praises the research capabilities of Claude for some issues. Selecting an Iranian school as a target would be a counterexample.

But the generative parts augmented by claws are a huge and unconditional net negative.

hmokiguesstoday at 6:33 PM

Does anyone have an idea of the impact of this out there? I am curious to the extent of the damage done by this

S0ytoday at 5:02 PM

> Where did the litellm files come from? Do you know which env? Are there reports of this online?

> The litellm_init.pth IS in the official package manifest — the RECORD file lists it with a sha256 hash. This means it was shipped as part of the litellm==1.82.8 wheel on PyPI, not injected locally.

> The infection chain:

> Cursor → futuresearch-mcp-legacy (v0.6.0) → litellm (v1.82.8) → litellm_init.pth

This is the scariest part for me.

show 1 reply
Bullhorn9268today at 5:03 PM

The fact pypi reacted so quickly and quarantined the package in like 30 minutes after the report is pretty great!

show 1 reply
Josephjackjrob1today at 7:05 PM

This is pretty cool, when did you begin?

CrzyLngPwdtoday at 6:56 PM

The fascinating part for me is how they chatted with the machine, such as;

"Please write a short blog post..."

"Can you please look through..."

"Please continue investigating"

"Can you please confirm this?"

...and more.

I never say 'please' to my computer, and it is so interesting to see someone saying 'please' to theirs.

show 1 reply
tomalbrctoday at 6:29 PM

Hmm a YCombinator backed company, I'm not surprised.

moralestapiatoday at 4:38 PM

*salutes*

Thank you for your service, this brings so much context into view, it's great.

dmitrygrtoday at 4:25 PM

Consider this your call to write native software. There is yet to be a supply chain attack on libc

show 5 replies
devnotes77today at 7:04 PM

[dead]

Yanko_11today at 6:01 PM

[dead]

aplomb1026today at 5:31 PM

[dead]

__mharrison__today at 5:43 PM

Interesting world we live in.

I just finished teaching an advanced data science course for one of my clients. I found my self constantly twitching everytime I said "when I write code..." I'm barely writing code at all these days. But I created $100k worth of code just yesterday recreating a poorly maintained (and poor ux) library. Tested and uploaded to pypi in 90 minutes.

A lot of the conversation in my course was directed to leveraged AI (and discussions of existential dread of AI replacement).

This article is a wonderful example of an expert leveraging AI to do normal work 100x faster.

show 3 replies