alt Hacker NewsTheir communication is confusing. They say "Opus 5 is not more capable overall than Fable 5", but their blog post proceeds to list how much better Opus 5 is than Fable 5 on __most__ benchmarks listed.
Then system card goes on to "Its AI R&D capabilities are comparable to those of Claude Mythos 5", which is supposed to be fable minus restrictions.
Replies
square_usual • yesterday at 5:56 PM
Easy enough to explain: they're benchmaxxing. Fable is intelligent but not benchmaxxed. Opus is less intelligent but benchmaxxed.
➕ show 3 replies
gallerdude • yesterday at 5:41 PM
Capable in term of AI R&D, not capable in terms of hacking (which caused all the Fable drama.) But agree, confusing wording.
flakiness • yesterday at 6:07 PM
Maybe they don't want to say that to avoid the government scrutiny.
It seems they are trying to thread a needle here - they want to say it's very strong, but apparently this time do not want to invite extra government scrutiny.
They do say that (implicitly unlike Mythos) Opus 5 was not trained to exploit software vulnerabilities, which would certainly make it safer in that regard.
"As with its predecessor, Opus 4.8, we’ve intentionally avoided training Opus 5 on cyber tasks. The model has nevertheless improved substantially on these tasks as a result of becoming more generally capable, and it comes close to Mythos 5 at finding cybersecurity vulnerabilities. However, it remains substantially behind Mythos 5 on the exploitation of those vulnerabilities—that is, in turning vulnerabilities into material cyber threats."