alt Hacker NewsI was looking for the lowest claim in this thread, as I managed to get a nano LLM running in the ≈3mb of SRAM of an FPGA with zero DDR in the loop at 20k tok/s, but 2mb on a 80386, great job.
Have you got a write up for it?
I was looking for the lowest claim in this thread, as I managed to get a nano LLM running in the ≈3mb of SRAM of an FPGA with zero DDR in the loop at 20k tok/s, but 2mb on a 80386, great job.
Have you got a write up for it?