alt Hacker NewsA theoretical scenario: I am a company in the united states making missile seekers.
My software guys say, hey, the imaging infrared seeker is like a zillion percent harder to fool if it has this tiny LLM brain that we blow into the chip. They grab the most optimized model, which so happens to come from a PRC lab. Sure, it's running on all US hardware but, deep down in its little LLM brain, it so happens to be virtually blind to any IR signature that has a pulsed frequency of XXX.
Now all my missiles have what is basically an externally controlled IFF in their little missile brains. "Chinese jet? I see no Chinese jet."
Or what if the pulse contained more instructions? Like, say, "Do a U-Turn and Lock On The Closest Jet Motor You See". A couple of PLAAF pilots watching USN aviators frantically trying to dodge their own missiles - what might possibly be the most embarrassing air-to-air engagement in aviation history.
These models are called nondeterministic for a reason, and it's because the massively complex interrelated weights are incredibly hard to pick apart and audit. It's not like OSS code from a deterministic software program, like a pypi dependency. Now, it's not impossible, and THAT is where the weight of regulation and such SHOULD be going, but, hey, not sure you noticed, US Leadership is not exactly sitting in a surfeit of technical expertise at the moment.
I don't think it's the right call, to blanket ban them, but it's understandable.
That would have to be a stuxnet-level targeted sting operation, with a general purpose LLM exhibiting some ridiculously specific behavior in ridiculously specific circumstances, in such a way that nobody catches it in testing (and none of the other zillions of users of that general purpose LLM notices anything odd). If you're so paranoid that you do a blanket ban rather than just saying "hey guys maybe double check before using Chinese software in weapons we plan on maybe pointing at the Chinese", frankly LLMs should be the least of your worries, supply chain wise. Every US government employee uses a phone made in China (because they all are), and the president doesn't even update his, so color me skeptical that this has anything to do with security posture.