alt Hacker News'Narrow scope produces better findings - Telling the model "Find vulnerabilities in this repository" makes it wander. Telling it "Look for command injection in this specific function, with this trust boundary above it, here's the architecture document and here's prior coverage of this area" makes it do something much closer to what a researcher would actually do.'
So what, we take every function and every vulnerability type and just run the agents millions of times?
I would expect Mythos to be able to find vulnerabilities without pointing it out for him, otherwise it's no better from other agents. It's just has a better harness.
Replies
I think the idea here is you give the Hunters (stage 2) a narrower scope, but have a parent agent responsible for dividing up the full search space (stage 1).
And note that Hunt tasks can be queued from previous Trace tasks, ie you find a vuln in one layer, so you queue a hunt for corresponding vulns in the layers that could exploit your first finding.
I'm still waiting something more specific or groundbreaking too. Feels like a lot of noise with just the goal to get people to talk about it. And now I realize I am talking about it and about nothing at the same time. Just fugazzi.
Yeah this whole post reads like Anthropic said “make sure you say how awesome Mythos is” but really what they’re saying is that it’s just a better harness.
This matches with what Nicholas Carlini from Anthropic said a the [un]prompted conference - https://www.youtube.com/watch?v=1sd26pWhfmg. Very worth watching.