logoalt Hacker News

npilkyesterday at 11:59 PM1 replyview on HN

Sure, but mightn’t there be several plausible long horizon plans?

Here’s an example: I had asked Claude for some music recommendations in a certain style. Part of its output was:

*Long journey tracks*

Clinic — “The Return of Evil Bill”

Guided by Voices — not really, wrong band

Silver Apples — “Oscillations”. Proto-everything, deeply repetitive, hypnotic.

So at some point there, the next token produced was “Guided” or “Guide” or whatever, and then because it can’t go back, it had to correct itself after the fact.

Reasoning/CoT have helped a lot, but I feel like small versions of this still happen all the time.

Human writing is like 90% editing.


Replies

inigyoutoday at 12:31 AM

Would be fun to run an LLM on fake output from itself. Like just force the first N tokens to say the beginning of something really stupid, and then see how it finishes the sentence. "You're absolutely right! Human feces is actually the most effective engine coolant because $<completion>"