alt Hacker NewsMore like lighting a fire in your codebase at the speed of light.
Nobody needs this, and the quality of models they're deploying, or all llms for that matter, require heavy human review (if you care at all about maintainability).
One developer can barely keep up with the daily output of multiple llm sessions writing code at 50 tps.
Im convinced anyone confidently thinking 2k tps is a no brainer is probably not concerned about the quality of their work (many such cases).
Replies
At those token rates you could treat the LLM as a compiler and rebuild your software from scratch every week. That would mean the end for AI slop because people who spend the effort to build a process that creates high quality software from scratch can take full advantage of the one shot performance of an LLM without having to release the process itself.
Occasionally it takes me at most 10 minutes to figure out that a solution the LLM took an hour to churn out is not up to par and I need to throw it away entirely.
There are diminishing returns, but half of the rate you mentioned would already help me keep focus.