logoalt Hacker News

ivansavztoday at 2:50 AM0 repliesview on HN

The style you refer to is the "container" of the writing. The medium. Like the specific encoding of the message. What @arctic-true was talking about was the "content" of the writing, which is bounded from above[1] by the information content of the prompt.

So, I'm not sure if it's a question of time at all: if a LLM text contains some piece of information beyond the information that went into the prompt, where does this "extra" information come from? [Note, I'm not thinking about facts which could trivially come from the training corpus, I'm thinking specifically as information in the sense of intended message from sender (author) to receiver (reader)]

[1] cf. this comment where I explain this analogy between LLMs and noise channel in communication theory: https://news.ycombinator.com/item?id=49510244