alt Hacker NewsNext sentence: "We expected the engine to return an error for any prompt it could not process, and our long-context tests returned successful responses, so we did not believe this to be an issue."
Replies
segmondy • today at 2:05 AM
Doesn't matter, they could have marketed it as 1 million token window, you would then infer, get to 600k and get an error. You won't be able to finish your work, and you won't get credit for 600k tokens.
➕ show 1 reply
freakynit • today at 1:40 AM
Bro, I had to read this sentence in the context of the previous one, 3 fcking times.. and it sill does not make any sense to me.
➕ show 1 reply
Which is pointing the finger at Sglang, who also weren't responsible for this.
At a minimum someone should have realized during testing that their utilization figures are off by 40%.
A much better title would be "Cloudflare configuration error leaked data between user sessions"