> It's also never deleted anything it shouldn't or done anything crazy or unrelated to what I asked

Until it does. A simple curl request to a compromised website could inject a malicious prompt into it.