alt Hacker NewsI like the illustration that the models are working on a convex hull of known information. Filling gaps with linear combinations of known facts and results.
They can't exit the hull until the "intuition" starts spawning points outside the convex hull.
Replies
bee_rider • yesterday at 9:20 PM
Is that actually true though? I think it is an analogy, and as an analogy it seems quite risky because “convex hull” and “linear combination” are technical terms that might give the recipient the impression that it is a technical argument.
jiggawatts • yesterday at 9:10 PM
Neural nets can extrapolate past their training data, and there is no reason to think LLMs don’t inherit this capability.
The extent to which they are able to do this is the more interesting question!
➕ show 1 reply
I think this is only "statistically" true in the sense that training is based on facts and not non-facts (except maybe with the ingestion of flat-earthers literature ;-). The existence of hallucinations in a bare transformer shows that the convex hull is not about information but about text, so the limit may more be "possible linear combinations of text", which allows for much extrapolation and counterfactuals. True creativity may be one reinforcement learning mid-training goal away that rewards novelty over correctness.