Wouldn't a local llm be just as vulnerable to this?

Well that as the set up is non-negotiable (it legally has to be on premises); the issue is a model nonetheless exfiltrating data if we give it any network access.