> This article feels like the result of having unlimited tokens at your disposal.
Gong to be honest where do you think this is going?
Do you think tokens are going to magically become more expensive or models are going to become less efficient?
Of course you should be tokenmaxxing, you can already tokenmax with a model that competes with the SOTA 1 year ago on a moderately powerful laptop you can walk out of an Apple store with.
This seems like a very strange comment to come from someone I am assuming is a SWE. When have we, ever, been happy with non-deterministic, objectively more expensive, solutions?
You're assuming a lot of costs that should just be considered ok in your stance:
* Why pay anything multiple times when you can replicate deterministically for free? * Why require always-online models when you can run computation locally? * Inversely, why use power-hungry/PU-hungry models locally when you can have a deterministic software that can be optimized to not* be memory/PU/power hungry to do validation?
I think tokenmaxxing can be a great start* to get things going but when it comes to productizing I want determinism and cheap optimizations, and using frontier models to build these cheap optimizations is what we've been doing at our company over this past year to great success.
To be a bit more grounded and expand on your point:
I, personally, have already experienced situations where I was employing cost-optimization optimization strategies for models 8 months ago, that are borderline-useless now because the Pareto frontier has advanced both more than I thought and also in different ways than I thought (cheap models got smarter, better at following instructions, and more clever about tool calls).
That is, the strategies decreased in effectiveness disproportionately to how much the model improved.
It's worth considering that strategies that seem insanely expensive now may be economically viable in a year or two. Or possibly significantly sooner than that.
It's also possible that, like how jet engines get more efficient as they get larger (leading to the 737 MAX incidents, which seems spiritually significant) corporations with budgets orders of magnitude greater than ours benefit from very different strategies than we do.