logoalt Hacker News

Vibecoding an inference chip

57 points • by ankitvgupta • today at 6:15 PM • 19 comments • view on HN

Comments

cpldcpu • today at 7:51 PM

I did something like this with Opus 4.5: https://github.com/cpldcpu/smollm.c

It seems implementing transformers is the new "hello world" of AI assisted coding.

datadrivenangel • today at 7:07 PM

Does not say what type of model or size they were serving, but one of the traces mentions Maple, which is a 20b model. Still, getting ~70+ tok/s is quite good.

➕ show 2 replies
codekansas • today at 7:30 PM

This is cool

Next step is tuning your post training pipeline to optimize your model for your chosen FPGA architecture...

chalkycrimp • today at 7:42 PM

Good chance I'm not the only one thinking this: share the code

lxe • today at 8:45 PM

Repetitive "make it faster" prompting is essentially what all these "RSI" claims are

spwa4 • today at 8:31 PM

I must say I do see one application for this, that you can't really match reasonably otherwise: you can bake weights into an FPGA and then run them for 5W at a speed that will be competitive with a big Nvidia card.

(unfortunately for a decent model it will also be price competitive with a big nvidia card)

saejox • today at 7:51 PM

i wonder if we can emulate an FPGA on a rtx 3090? then test it on an AWS EC2 F2 few hours a day.

Archit3ch • today at 8:24 PM

Now do silicon. ;)

➕ show 1 reply