alt Hacker NewsIt kind of blows me away how “unreasonably effective” these small image models seem for their size.
Meanwhile a decent LLM that can do tool use and code and reason tends to weigh in at a minimum of 30B parameters, and you need over 100B to get good performance.
Do LLMs just contain more information or are diffusion models more efficient at compression?
I saw some work on language diffusion models a while back but I think they tend to have problems with reasoning versus transformer type models.
Replies
To make a rather speculative argument - since an LLM should be able to describer an image losslessly, images are a subset of what an LLM needs to understand. It is possible that the statistical model being learned for images is just simpler.
The get a proper LLM they have to model all of human knowledge regarding patterns, physics, biology, social interaction, skills, history, etc, etc. For images they have to recall what a dog looks like in a still photo. The image models aren't even necessarily good at a wide range of novel situations; usually they get used for stock scenes.
Natural evolution has also had a much easier time coming up with vision models (everything with eyeballs) than complex language models (as far as we know only a smallish family of monkeys).
At first, I considered that diffusion models struggle with reasoning because they underweight 'prior beliefs', ie the propositions by which an argument advances.
Then I considered whether the universe of imagery is simply smaller than the universe of ideas—which must, itself, include the universe of images, too.
But finally, I'm thinking maybe images are just harder to benchmark.
Or maybe they're _not_ actually worth 1,000 words.
They can make good images but they all look the same. Guess it's not hard to replicate a small section of what exists.
[flagged]
These models are, in a sense, distillations of language models. They wouldn't be as good without a pre-trained language model, so it's unfair to compare them.
Image models are essentially a mapping of text embedding space -> image. So the clearer and cleaner the text embedding space is the better model you get as you need less weights trying to interpret a text prompt, recent image models even use multimodal versions of LLMs (Flux uses Qwen3 VL) so all the visual separation was done beforehand making the model even more efficient.
edit: Maybe a better analogy is that the image models are only half the model in a encoder-decoder sense, where the LLM is a pretrained frozen encoder and the image model is the decoder.