logoalt Hacker News

johntashyesterday at 11:28 PM0 repliesview on HN

Can you explain how context fits into this picture by any chance? I sort of understand the vram requirement for the model itself, but it seems like larger context windows increases the ram requirement by a lot more?