logoalt Hacker News

jacquesm • yesterday at 12:38 PM • 0 replies • view on HN

Yes, because I've noticed that anything at all that is in the input stream gets used to waste tokens on various rabbit holes. Less information -> more efficiency and better output as long as it is sufficient to get the job done. For a human distinguishing important stuff from unimportant stuff is hard for an LLM it is next to impossible (at least, from what I've seen so far). There is no distinction between 'obscure thing that I once saw in passing 30 turns ago versus 'the thing right in front of my nose'. So you get all of these wild goose chases that lead nowhere and that will easily burn 50 to 100K tokens before finally being abandoned, more often than not after doing considerable littering and/or actual damage. So I am purposefully limiting what goes into the LLM and I find that by doing this it gets to the point much faster and produces better output. 'Lead it not into temptation' or something to that effect ;)