logoalt Hacker News

cheriooyesterday at 11:30 PM1 replyview on HN

Thinking generates a ton of tokens. These baked in chips tend to not have a lot of memory for context. I am not sure taalas supports Thinking at all.

The more problem like these they solve the more they will look like GPU.


Replies

nextaccounticyesterday at 11:43 PM

couldn't one just add some hundreds of GB of HBM?

show 1 reply