I think the main problem is surfacing two topics:
- Understanding how effort and class interact (Is a high effort terra above a low effort Sol? Are larger models mostly always better regardless of effort?)
- understanding when (if ever) a new model tends to be better than an older but higher class one. Is an old astra better than a new terra?
If increasing effort and increasing model class produce improved results in a different way, I miss making it explicit in which way. In other words: What kind of issue warrants going up in effort? what kind of issue warrants going up in model class? What kind of issue is unlikely to be solved by either?
>effort / performance curves by model
Oh, easy:
Artificial Analysis has their own curves which are pretty similar.
> Is a high effort terra above a low effort Sol
You can literally see this in the charts from the article
OpenAI itself cannot answer these questions. A prospective price-sensitive (or speed-sensitive) user has to experiment to find the cheapest (or fastest) model that is sufficient for each task.