logoalt Hacker News

datadrivenangel • today at 7:07 PM • 2 replies • view on HN

Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.


Replies

risyachka • today at 7:23 PM

Fpga is software-driven, model can verify its output easily.

Zero surprise it did it

datadrivenangel • today at 7:08 PM

it only cost them $10,000 in Claude API usage. :)

➕ show 1 reply