If you optimize program A and manage to wring out a 1% improvement, and I optimize program B and improve performance by 20%, you can see the problem with trying to infer anything from those two numbers.
Edit: searching for the story now, further bolstering the point is that was 1% in training time [1], and the openAI claim is 20% in end to end inference cost. This is a bad comparison.
If you optimize program A and manage to wring out a 1% improvement, and I optimize program B and improve performance by 20%, you can see the problem with trying to infer anything from those two numbers.
Edit: searching for the story now, further bolstering the point is that was 1% in training time [1], and the openAI claim is 20% in end to end inference cost. This is a bad comparison.
[1] https://deepmind.google/blog/alphaevolve-a-gemini-powered-co...