Look at the "Q4_0 Prompt" column. That's the tokens-per-second processing the initial prompt/system prompt. This is where the "neural accelerators inside each GPU core" is seen most prominently.

The M5 GPU added better matmul and dot product support in hardware (IIRC) that really boosted performance of those kernels over previous GPUs. Current transformer models rely heavily on matmuls. Prefill (processing the current context) is mostly limited by raw GPU processing power where token generation (predicting the next token from the current context) is mostly limited by memory bandwidth.

While an older M-series might run a matmul or dot product kernel just fine the M5 can run them much faster.