logoalt Hacker News

tedd4uyesterday at 10:02 PM0 repliesview on HN

Look at the "Q4_0 Prompt" column. That's the tokens-per-second processing the initial prompt/system prompt. This is where the "neural accelerators inside each GPU core" is seen most prominently.