To be fair, “s’kiddies will exploit low hanging fruit with semi-automated vuln scans” is a lot more realistic threat than “LLMs are going full Skynet any day now!”
I’d definitely suggest companies start addressing that first concern, even if I’m in the camp that thinks the second is fantasy.
I think, realistically the way Skynet happens is that the military abuses LLMs for warfare. The military is historically, and currently, incredibly irresponsible with technology. If we aren’t already automating killer drones and such, then we will be soon because “well our enemies are doing it!”
I don’t think that means that it’s gonna, like, somehow homogenize into some mega super intelligence. But we will have machines who are designed to kill, and do so without human input or alignment.