alt Hacker NewsWhat about inference providers like Baseten, Modal, Fireworks, Together, etc? I thought one of their value propositions was inference (using open weights models) that guarantees with crisp terms that they will not use your data.
Replies
A provider can genuinely avoid storing inputs, as the Baseten engineer below describes. That is still different from proving what code received the prompt or protecting plaintext while it runs; I built TrustedRouter to separate ZDR, attestation, and confidential routes: https://trustedrouter.com/blog/attestation-is-all-you-need?u...
> using open weights models
AWS and Azure give you the same thing for Claude and ChatGPT, no need to be stuck with open weights. They might sometimes store some of it for other purposes (I don't know the specifics), but it is emphatically not being fed back to OpenAI or Anthropic.
I don't have any much exposure to the attitudes people have around them, and I haven't worked with them. So I can't really say
I worked very briefly at Baseten, and I can say that it was a perpetual annoyance (from an engineering perspective) that customers would complain about issues with their models but we couldn't actually see the inputs/outputs. I don't know about the other providers, but at Baseten they literally weren't stored anywhere.