alt Hacker NewsAny sources on training costs in free fall? My understanding was training costs are proportional to model size. Is it just that compute costs are falling?
Any sources on training costs in free fall? My understanding was training costs are proportional to model size. Is it just that compute costs are falling?
MiMo RL'd their latest models for $3.5M (million, not billion). It's hardware, software, and algos
https://mimo.xiaomi.com/rl/
Also look to the history of stable diffusion