logoalt Hacker News

anon373839yesterday at 11:11 PM2 repliesview on HN

The models are post-trained on these prompt additions so they’re more structural than thinking of them as “system prompts” suggests. (All LLMs ever see is tokens going in, so even the concept of a system prompt is just formatting they’ve seen in post-training.)

You can also apply fixed token budgets for the reasoning blocks, though it will decrease quality in some cases.


Replies

nixon_why69today at 1:05 AM

Why not invent a few magic token values for reasoning level instead? It would be like 4 out of a vocabulary of 200k and save like 30 tokens in every prompt

thousand_nightstoday at 1:00 AM

yes of course, I understand that. but I feel like it would've been nice to include in the article because the main point of it is the effort and overthinking