You dramatically overestimate how much time engineers at hypergrowth startups have on their hands
There's a direct business incentive to game/cheat benchmarks, it wouldn't even be difficult to do, and besides, they have workforce-replacing AI to do it for them.
Caching some data is time consuming? They can just ask Claude to do it.
There's a direct business incentive to game/cheat benchmarks, it wouldn't even be difficult to do, and besides, they have workforce-replacing AI to do it for them.