logoalt Hacker News

hirako2000today at 7:10 PM0 repliesview on HN

They are. Empirical evidence on my side. Because attention is sparse across the context. It's not truly treating a million token the way it treats a fraction of that count. For performance.