logoalt Hacker News

hattmalltoday at 3:03 AM0 repliesview on HN

We are seeing more of their "thinking". Lowering the refinement of the output to get closer to profitability. The nature of the LLM is that it generates huge amounts of text, then it iterates them down into a compact, hopefully accurate prose. That refinement is the really hard part and computationally costly.