alt Hacker News1. That sounds like an Anthropic concern, not a concern for most customers who don't run their own model. 2. This still requires trust in the provider, I imagine these proofs would change weekly as they tweak their models, and you have zero knowledge of what the changes are, you just know you're getting version 11.12.1999 - does that really help you if the provider is maliciously downgrading after release? All you know is you got the same version as everyone else, not what that version is. 3. Again, if you don't trust the provider, it's game over - you're sending them all of your data. Knowing a version number is not going to help you much.
I'm not convinced this is useful for problems most people face using models, it seems a very niche academic concern frankly and an incredibly complex system to prove not very much.
Replies
>1. That sounds like an Anthropic concern, not a concern for most customers who don't run their own model.
You're talking about this as if a bug or defect in their inference stack has no impact on you as the customer.
1. It's a provider concern and a developer concern. If your infra is serving broken proofs that might be justification for compensation for breaking SLA.
2. Models only change weekly for closed frontier models and even then they should be reporting model "sub-versions" if only for observability purposes.
3. This doesn't really hold true. Kimi and DS release their models on a monthly or bi-weekly cycle and while it's still DS-v4.1-Flash or whatever, under the hood it's DS-v4.1-Flash-1017 or some other date code identifier of which specific model it is and that is both exposed to the user and that model version can be requested explicitly by the user.