alt Hacker News> Right now, overall compute is a moat.
Very true. Or further, access to capital is the moat. It is the very reason that we don't have a real open-source community that trains frontier models - individuals simply can't afford the training infrastructure, nor sufficient high-quality training data.
https://hugovergnes.github.io/little-lm-3-8b/ << less than $1k for a 4B model
Remember how DeepSeek v.whatever cost ~$5m
Stable Diffusion 1.5 was reportedly $70k in compute.