alt Hacker NewsThis is not an LLM obviously , it's just for generating random names. But interesting to think of the possibilities of truly tiny language models if there were connected together.
Replies
nomel • yesterday at 5:33 PM
Hasn't it been repeatedly shown that many small models perform worse than a large model of the same total parameters?
➕ show 2 replies
elzbardico • today at 2:08 AM
LLMs are an imprecise, more of a marketing term, to define Transformer models based on the self-attention mechanism, trained with massives amounts of data.
And this implements a transformer. Actually it is a very cool didactic example.
api • yesterday at 4:26 PM
Isn't a MoE model basically a cascading tree of smaller models or some variation of that?
➕ show 1 reply
It's an slm (small language model) due to number of parameters and it uses the same architecture as an llm, but llms have billions of parameters