alt Hacker NewsThis reminded me of Gaussian Processes. You start out with n-dimensional unconstrained (but strongly correlated) gaussians. As soon as constraints (data) are added (mathematically it's called conditioning), the thing goes more and more into shape.
Prompting feels a lot like this conditioning phase to me. You start with an LLM in unconstrained mode, basically just a "soup" of knowledge. If you prompt wisely, you immediately condition the LLM into "your space of (domain) knowledge".
What comes out is an extended version of your existing knowledge.
Replies
Culonavirus • today at 2:29 PM
It's this https://youtube.com/shorts/OPVp_FDEkdA
Yes I love this comment! GPs are awesome.
Prompting is conditioning, that is what it is. The visual of a GP (like the thing you get if you google image search “Gaussian process”) is a great metaphor for what prompting an LLM is doing.
The output of the LLM is the logits which is sampled - plucking out tokens from a distribution. The input of the LLM is data which constrains the logits. That is what it is.
That’s also how you know that AI will never “solve” intelligence (the way the boosters say it will) without some general mechanism for this conditioning process. The ultimate mechanism would be embodiment; the crappy mechanism we have now is something like openCLAW.