A quick look at Mythos run on Firefox: too much hype?

There was a double fronted marketing push by both organizations. That much is true and this makes me more skeptical of the message and how exactly it was framed.

If we just stick with c/c++ systems, pretty much every big enough project has a backlog of thousands of these things. Either simple like compiler warnings for uninitialized values or fancier tool verified off-by-one write errors that aren’t exploitable in practice. There are many real bad things in there, but they’re hidden in the backlog waiting for someone to triage them all.

Most orgs just look at that backlog and just accept it. It takes a pretty big $$$ investment to solve.

I would like to see someone do a big deep dive in the coming weeks.

Probably worth noting that the new-ish Mozilla CEO, Anthony Enzor-DeMeo, is clearly an AI booster having talked about wanting to make Firefox into a “modern AI browser”. So, I don’t doubt that Anthropic and Mozilla saw an opportunity to make a good bit of copy.

I think this has been pushed too hard, along with general exhaustion at people insisting that AI is eating everything and the moon these claims are getting kind of farcical.

Are LLMs useful to find bugs, maybe? Reading the system card, I guess if you run the source code through the model a 10,000 times, some useful stuff falls out. Is this worth it? I have no idea anymore.

The author seems to believe that dereferencing a null pointer is safe. DoS attacks aside dereferencing a null pointer in C++ is undefined behavior so you never know what could happen. It could easily result in bypassing seemingly unrelated security checks or any other behaviour. To know it wasn't exploitable you would need to check the compiled output of every compiler and set of flags used to compile Firefox.

Why people publish AI written articles? If I would like to read AI I can just prompt it myself, and when I read something on someone blog I expect that I will read thoughts of this particular human being...

It’s just marketing. Remember when OpenAI said GPT-2 was too dangerous to release?

IIRC Mozilla usually categorize internally-found bugs into a few large CVE IDs, grouped by severity, with around ten or so bugs in each. Every advisory gets several CVEs of this kind, for example, <https://www.mozilla.org/en-US/security/advisories/mfsa2026-2...>, <https://www.mozilla.org/en-US/security/advisories/mfsa2026-1...>, <https://www.mozilla.org/en-US/security/advisories/mfsa2026-0...>, etc.

This article felt really informative at first but sone point it was like reading an LLM getting stuck in a circle

One think to keep in mind is that firefix is probably a pretty hard target. Everyone wants to try and hack a web browser. One assumes the low hanging fruit is mostly gone.

I think the fact this is even a conversation is pretty impressive.

Whatever the capabilities, there’s always a little hype, or at least the risk won’t be as great as thought:

> Due to our concerns about malicious applications of the technology, we are not releasing the trained model.

That was for GPT-2 https://openai.com/index/better-language-models/

Can IDE's be configured so that it won't allow to save the file changes if it contains the usual suspects; buffer overflows and what not. LLM would scan it and deny write operation.

Like the Black formatter for Python code on VSCode, that runs before hitting CTRL+S.

There's a pretty big problem here, which is that all of the security bugs (the serious ones) are embargoed. So going off of public info is not really useful

> Conclusion

"The Firefox 150 data suggests a tool that is genuinely useful for defensive security work, especially at scale, but the public record does not justify the strongest claims people want to make from it. The headline number is impressive, yet it bundles together bugs of very different significance and does not publicly resolve into a clean accounting."

I mean: Obviously. Does not matter how good or bad a product is, the current meta is to over-hype it in order to achieve maximum "news-penetration". Anthropic seems to have sth. "real". However, Since there is no way for outsiders to calculate real metrics like false-positive rate, cost (tokens, Dev hours for setup and review, ...)/ Issue found, ... there is no real way to put any scale on the hype-graph.

For crying out loud, why are we discussing and paying attention to articles and claims about a product that doesn't even exist yet?!

If this isn't a sign of a bubble, where marketing is more important than the actual product, I don't know what is. This industry has completely lost the plot.