logoalt Hacker News

m12ktoday at 7:18 AM3 repliesview on HN

So we've basically taken the concept of branch prediction from CPUs and applied it to LLMs?


Replies

mike_hearntoday at 8:57 AM

Maybe at very high level of abstraction, but there's no branching involved.

show 1 reply
c7btoday at 7:32 AM

The concept of predicting future elements in a series is not specific to CS. It's older than computers.

fragmedetoday at 7:23 AM

Well, the TPUs they're running on don't have branch prediction, so that had to end up somewhere in the stack.