alt Hacker NewsReally almost all benchmarks I look at have a cost per task column, which is basically the code size metric if you take an extra step
Really almost all benchmarks I look at have a cost per task column, which is basically the code size metric if you take an extra step
Not at all. The model could (and sometimes should) burn all the money it wants, and then produce a single line of actual production code. Only some things, e.g. full rewrites, have clear cost - LoC scaling.
For my usage, I would very much prefer if those $/task were being spent in thinking and experimenting, and the actual output would be as short and maintainable as possible. “maintainability” is a vague target of course, but it’s at least somewhat correlated with code size.