alt Hacker News> frequency bias
Gaslighting again, Anthropic shill. What does frequency bias have to do with the objective fact that hundreds of people reported their own experiences with LLMs being degraded over a short period of time? The very same tasks that the very same LLMs could do, they no longer can? You seem to ignore this FACT, this DATA, and instead have to gaslight and divert into "frequency bias" nonsense. I do understand, why you are doing it, Anthropic shill, but at least have guts to admit it.
> perception of a model
You once again ignore what your LLM outputted and you typed yourself and divert into "perception", Anthropic shill. You do not need to sample entire output for tokens to notice the distribution moving. If the LLM used to be able to achieve set goals and no longer could, it is already a sign of the distribution shift. And as you said yourself, different token distribution = model being quantized. Which is reported in hundreds of separate instances. Which is more than enough to conclude that the model was, in fact, quantized, and no amount of gaslighting can change that. But you are an Anthropic shill, so you have to peddle your bullshit, trying to twist facts to support your employer's narrative. And you deserve being called out on that, Anthropic shill.
> What does frequency bias have to do with the objective fact that hundreds of people reported
This isn't hard to understand
"Model is nerfed" claim hits social media
Someone else sees it, frequency bias makes them think their model is also nerfed, and they amplify the claim
Now it spreads, like a virus, even if the model never changed
Social dynamics like this are well understood psychologically
> If the LLM used to be able to achieve set goals and no longer could, it is already a sign of the distribution shift.
The more likely explanation is that you're looking at older LLMs with rose tinted glasses, and misremembering what it could achieve
Otherwise you could measure the token shift and see the better tps and latency
Your own evals would trend down
But no one, not one person, has presented empirical evidence of being served a quant. Just vibes.