logoalt Hacker News

Forgeties79 • yesterday at 4:25 PM • 2 replies • view on HN

I agree their phrasing is a little strange but I feel like the meaning is still clear. They are (I believe) saying that an LLM can’t generate anything other than facsimiles of existing content. It can’t actually explore novel ideas and branches, it can’t innovate and create on its own. Everything is inherently derivative yada yada, I don’t mean it in the usual “everything is derivative” way we often refer to when people make music and such. It can’t ideate and express something personal.

I think there’s a lot of nuance and caveats to that kind of claim, assuming I am representing them correctly, but it’s a valid stance and I see where they’re coming from.


Replies

adrianN • yesterday at 4:50 PM

All possible books live somewhere in L-space (ook). Exploring that space efficiently is equivalent to creativity. The claim is that the space that LLMs learn is somehow sufficiently limited that it doesn’t contain novel ideas? I never saw a convincing argument why LLMs output can not be considered creative.

➕ show 3 replies
basch • yesterday at 4:43 PM

Seems like this might be true given current implementations but not ANY implementation of the technology.

If we’ve mapped all of English, there are still ways to construct stories that haven’t been told from the mapping. The most simple would be a subtraction of all objects minus all objects that have ever been used as a murder weapon. Leaving you with a list of never before used objects at your disposal.

There are also probably close vectors conceptually but not in semantic use. Or trying to interpolate or extrapolate what exists in voids of space. Turning up the temperature and adding random generation of words. There’s tons of ways to tune these things to be more creative than regurgitative.