alt Hacker NewsIt's not quadratic attention, you get that curve from the input tokens going up linearly, since the graph is measuring cumulative cost at each token count. Basically for y=5 it's 5+4+3+2+1, or f(x) = x(x+1)/2
It's not quadratic attention, you get that curve from the input tokens going up linearly, since the graph is measuring cumulative cost at each token count. Basically for y=5 it's 5+4+3+2+1, or f(x) = x(x+1)/2
On the one hand, compacting at 300k saves OpenAI 40%. That's great.
On the other hand, $10 for 1M tokens still seems really high? It's not too hard to blow through that in an hour or two.