alt Hacker NewsToken use exploded a week or two ago. I worked for a few hours and hit my 5 hour limit. Did only some simple bug fix. Somethings definitely broken.
This is the #1 thing i HATE with LLMs. They are not predictable like a compiler is.
Replies
It's probably not token usage, but a flexible 5 hour limit. There's a reason why they only state it in percentages.
They're actually extremely predictable. What you are experiencing is the provider you are entrusting with your time, effort, and money fucking you because they need the capacity to train the next one. So without any real notice or indication other than things vaguely seeming broken in ways everyone seems to be experiencing differently, they're serving up some shitty quant of what you already paid to use.
It's not LLMs! Its their providers! If you self-hosted, you'd observe the slow down but you'd get same quality.