logoalt Hacker News

caxap • yesterday at 6:33 PM • 1 reply • view on HN

As much as I love the idea of Taalas, their chatjimmy demo is a hard-wired (and probably very quantized) 8b LLM producing garbage extremely fast (17k tokens/s). Extrapolating from the picture of their PCIe board in their demo, I would expect a frontier model to require an IC not smaller than my desk.

At these dimensions, you are better off producing general-purpose inference ICs that can run any model, instead of baking in a single fixed LLM.

So, Taalas (now AMD) will probably target small models with these (maybe something like qwen3.8-27b, which seems too large already), if at all.


Replies

FuriouslyAdrift • yesterday at 7:00 PM

The long term target is JEPA-like primitives so models become irrelevant. You only have to accelerate critical paths to see massive gains.