I noticed a similar issue going from Opus 4.6 to 5.5. I also however upgraded my harness during that time, and I found out that it was injecting something like “Be helpful, read in between the lines and proactively go beyond the user’s request” at each turn. I immediately patched that out and saw decent improvement in that area. TFA mentions they tried different harnesses though, so yeah it’s prob baked in to whatever RL setup they’re using to train the original model.
This is why I feel like instruction following is really all I care about in a LLM. I just want it to do what I asked (not what it thinks I actually meant). Intelligence can always be supplemented with tool calls, but an LLM that won’t do as it’s told is no good
This. Yeah I think if one knows what they are doing, e.g. a seasoned developer. They can get a lot of mileage out of an LLM that is good at instruction following. which is why i like codex models. Astra may not be as good as Opus but the workhorse really is 6.2 sol. The best workhorse when it comes to the pareto frontier of `cost vs accuracy` or `inference time vs accuracy`.