alt Hacker NewsWe studied this question pretty systematically in the MirrorCode paper [1], comparing Python, C, Rust, Go, OCaml, and Ada across 19 very long-horizon tasks, for Claude Opus 4.7 and GPT-5.5.
> In our results, there was little sign of inter-language differences in solve rates, for any model (Figure 5b). This suggests that AI models have learned generalized programming skills, rather than pattern-matching syntax. This does not mean that implementation language is irrelevant. Conditional on solving a target, we found a small effect on token usage: successful Python solutions tended to use fewer tokens than average, while successful Ada solutions tended to use more (Appendix C). We consider these to be small differences, given that these six programming languages vary widely in how concise they are, and in how much functionality is provided by their standard library (recall that agents cannot download dependencies in MirrorCode, they must solve the task using only the standard library).
In Appendix C, Ada tended to use only about 25% more tokens than the average language. Ada is a language used mainly in safety-critical aerospace and defense systems, which has ~200x less pre-training data available than C or Python.
We're also comparing more recent language models (on just Go vs Ada, for cost reasons), on our leaderboard [2].
Reading your quote really gets me wondering who the people making those kinds of analysis are...
It's like they haven't maintained any actual software, because the criteria they choose is... Completely irrelevant?
The things that matter are tooling, orchestration and ecosystem - as well as how the LLM will actually implement the solution for a task
LLMs constantly do idiotic things. If you have good libraries to utilize, the likelihood of the solution actually working goes up because they no longer need to implement the hard part.
If you have orchestration for dependency injection, code generation, meta analysis etc
Tooling like the way otel tracing is integrated, openapi generation etc is also invaluable because every time the LLM does something the likelihood of it being hallucinated/wrong increases etc
You'd need to implement a nontrivial project in different languages, then add nontrivial features across them and only then start by rating eg correctness and incident occurrence after the final output
But token use on a one shot? Completely irrelevant as far as I see it.