alt Hacker NewsChinese labs seem to think that fable/mythos is a 10T+ model, and target that amount of parameters with next iteration of qwen/kimi/ds/glm. Proves nothing tho.
Chinese labs seem to think that fable/mythos is a 10T+ model, and target that amount of parameters with next iteration of qwen/kimi/ds/glm. Proves nothing tho.
With common use of n-gram tables, comparing "params" count has gotten harder. Those are exceptionally sparse and light on compute, way beyond even the sparsest MoE experts. You could host them on a slow spinning disk and still have a viable local inference platform.