logoalt Hacker News

reliabilityguyyesterday at 7:24 PM2 repliesview on HN

How exactly the map-reduce will happen though? Won’t you need to do it host-side, or make a lot of reads and writes?

Also, doesn’t it mean that you forgo batching?


Replies

ACCount37yesterday at 9:28 PM

Map-reduce is implemented as a rolling calc, see: online softmax in FlashAttention kernels.

show 1 reply