alt Hacker NewsLLMs are bad at creative work and I don’t see them improving any time soon. Try asking an agent to write a story about raccoons. It will almost certainly involve either stealing food or raiding trash with a 50% chance of having a character named Pip. If a location is mentioned, it’ll be Elm Street.
I guess this is the average story and, similarly, the average game is boring and predictable
Replies
Heh, tried it and got a story about raiding trash starring Pip. But yep, I'm well aware of this problem as well. /r/sillytavernAI is a sub dedicated to trying to coax LLMs into good creative writing but it's kinda impossible to do consistently.
I don't think this is bad. I want tools that deliver what I envision. The fact that given the same input, I get similar output is something I would rather consider a positive. The imagination should come from the human at the wheel.
While they are bad at creative work*, this reasoning isn't going to show it.
If you took the best, most creative, human writer in the world, and for thought experiment reasons they had amnesia (to mimic AI blank context windows) specifically while you asked them for a story idea 100 times in a row, my expectation is that this human would also give you the same idea at least 80 times out of that 100.
* still better than the mean human, but even the top 0.1% of humans aren't all professional authors.
Seriously, what is the deal with Pip! I was experimenting with childrens' fiction more than a year ago and 9 times out of 10 you'd get a character named Pip unless you were specific in the prompt not to do so.
1/3
I did get pip, but it's a story about trying to rescue the moon from a lake and the raccoon lives on moonberry lane
What's pip from?
I think the trick is to use LLMs like scalpels instead of hammers. That said, you can place a scalpel in the hands of a deterministic (or not) robot.
Elias Thorne!
Ironically, they're getting worse at creative work because all the RL is collapsing their distributions.