Benchmaxxing via memorization is boring and doesn't fool anyone for too long. It works, but then new benchmarks test old models and the real results fall in line. Benchmaxxing by focusing on specific types of things that benchmarks test on, while still not improving intelligence or capability in the general case? Not only is it blatantly obvious that all AI labs do this, but it's not even obvious how you would go about it any other way.

Now I am not really specifically accusing Anthropic of anything here, I'm just saying their behavior is suspicious. Since you tested Fable, they wouldn't even have to lie to have optimized for your specific benchmarks, since they absolutely had permission to read your sessions if they wanted to. But obviously, that's only the situation if we take them at their word. Personally I would be a bit surprised if they just flat out were lying and secretly retaining data they say they are not, but not that surprised. The penalties for doing this are probably worth the rewards if it keeps them super far ahead in the benchmarks for years without anyone catching on.

(In actuality though, even if they really were trying to sneakily grab samples of benchmark tests via their Fable data retention rules, I don't really suspect there would've been very much time to optimize Opus 5 on it. So consider me bothered.)