logoalt Hacker News

anemllyesterday at 2:28 PM0 repliesview on HN

Right.I was thinking about it, you still need batch refill, however, Apple Core ML tools were failing for attention activations quantization. Long context, pre-fill is still compute bound.