alt Hacker NewsThe fact it needs to implicitly be stated.
Every prompt will get this info, probably not only initially, since it could run out of context window. But every re-prompt in existing chats, even, gets one system prompt per model used per chat dialogue done.
Adds a fraction of cent of electric power just to every usage.
At home, I work with 65k context window, and if my system prompt and agents.md were both this length, I would spend two-thirds of the input window, before compacting which perhaps alleviates the issue for my use case, on re-feeding what mostly the reinforcement-learning should have implicitly baked in.
Replies
Because everybody has the same system prompt, the KV caching will make this a non-issue. The only cost is the reduced max context length.
But what makes it crazy? If nobody told you it was 22k chars, and it gets the provider the results they want for their benchmark goals, why is it crazy?
The only observable side effect, as a user, is that the system does what you want more often than one without this prompt. Or, it stays more aligned with the provider’s guidelines. Or some combination of both.
If it was a 65k context window, then it might be a bigger deal. But it isn’t, so the comparison is moot.