alt Hacker NewsMeh, I doubt it was intentional. Deliberate benchmaxxing is incredibly damaging to credibility once it's discovered (see what happened to Meta with LlaMa 4).
It's more likely that the training data was contaminated with the benchmark data.
Meh, I doubt it was intentional. Deliberate benchmaxxing is incredibly damaging to credibility once it's discovered (see what happened to Meta with LlaMa 4).
It's more likely that the training data was contaminated with the benchmark data.
You really think they saw the jump in arc-agi-3 (which they reported in their official card), and didn't even bother to check?
They maybe have not intentionally benchmaxxed, but they certainly know that's what happened .