Aren't there already much more efficient ways to make inference private?
Using regular encryption and secure enclaves, there are already providers that are roughly 2x the cost of normal providers. For example, https://tinfoil.sh/
Yes, by running models locally
If I used regular encryption to send my credit card information to an AI with fraud detection, the provider still needs to decrypt that data on their side at some point before it goes into the AI.
Using this other encryption, the provider has neither need nor capability to decrypt it on their end, so the user gets extra security.
Check out opaque.co
There's a much more simple and much more efficient way to do it: Policy and Legal contract.
> aren’t there more efficient ways
Heh yes absolutely, but there is some nuance.
Secure Enclave still requires you to trust the operator and also trust that it’s configured properly, supply chain is secure, etc.
The beauty of FHE is that it doesn’t rely on the compute being secure. All you need to secure are things you already have control over as a client.
I agree with you it’s still way too slow to be generally useful. (By general, I mean practical for arbitrary computation — you can relax the requirement and have fast homomorphic encryption if you only do specific kinds of operations).