logoalt Hacker News

nwlieb • yesterday at 8:40 PM • 2 replies • view on HN

I'm personally hopeful that as an industry we can move towards E2EE and attested secure enclave pipelines to remove the "actual knowledge" argument. NEAR AI and Tinfoil are the only publicly available providers in this area but I would be excited to hear about more advancements in this area because I personally think that these NCMEC integrated "ZDR" systems are major privacy violations and deceptive marketing.


Replies

AlanYx • yesterday at 9:31 PM

Azure does offer nVidia's secure enclave architecture via "Azure Confidential GPU", but it's limited to a single GPU and a little less than 90GB after overhead, so basically maxes out at Mistral-Large-2 with 4 bit quantization.

This whole thing kind of puts the industry in a bind. On one hand, the big labs do have an incentive to get this kind of thing rolled out so as to avoid open weight on-prem AI gaining traction, and to remove remaining roadblocks to corporate adoption in regulated industries. But rolling it out risks attracting even stronger regulatory responses for them as well (that's even setting aside likely national security/law enforcement pushback), as well as likely being strongly opposed by the AI safety people. There's no one way to square the circle that will please everyone. My guess is the labs will take a third route: continue to lobby against open weight models, thus killing the possibility of on-prem, and then drag their heels on E2EE/secure enclave models except perhaps for military customers.

➕ show 1 reply