alt Hacker NewsKeep in mind that the model is thinking in a token space, itself a compressive representation of language.
(Note: there's still a huge grammar penalty, so, ugh do think small.)
Replies
kzrdude • today at 7:10 AM
It selects tokens but they expand to embedding vectors which are huge, also in memory and attention requirements, I think?
The real breakthrough is going to be thinking in latent space.