You care about open models and you project that care on to the world and your rationalization of it. In reality open models are a thing, but not the biggest issue. Open/closed whatever the advance of capabilities is the real issue people are concerned about.
I’m not afraid of AI, even AI with extreme abilities.
I am afraid of humans with AI. That’s because I’m afraid of humans. When I look around the world I see humans murdering and robbing each other. When I get online on almost any social media I’m confronted by a wall of hate and grievance.
I am afraid of what humans will do with AI, and one of my biggest concerns there is what happens if small groups of powerful humans are able to monopolize it and leverage it against the rest of humanity. The rest of humanity can’t defend itself by leveraging it back because they don’t have it. It was kept from them for “safety.”
If the AI itself starts showing emergent volition and acting independently, I find that less scary. Not zero scary but more a mix of concern and fascination. The human monopolizing AI scenario makes me wish I had a weapons stockpile, as if it would matter.
I’ll explain it like this. Two options.
Option 1 is you wake up to find a grey alien or a ghost in your room, something alien and beyond your understanding.
Option 2 is you wake up and find some random guy in your room.
Which do you pick?
I pick option 1. I’d be afraid if I saw something paranormal, but in a less visceral way. I see a dude I’m going to start scanning the room for anything I could use as a weapon.