logoalt Hacker News

ranyume • today at 2:08 PM • 1 reply • view on HN

Even with a harness, there are things LLMs can't do. e.g. Write 50 entertaining short stories, each one completely different from the other but all sharing one motif.

In the example, LLMs choose the shortest path that would be verifyable by an evaluator. So the model is essentially blind to "entertaining" and "completely different". The LLM will output stories with different variations of the same structure.

If the LLM can somehow read other works and derive from them, it will merge the motif with the story being read.


Replies

brainwad • today at 5:43 PM

That task sounds like something that at very least most humans couldn't do either and possibly is outright impossible (completely different yet sharing the same motif?).