There’s just also so much confusion about LLMs and Agents because they use language outputs and can do these higher level reasoning tasks, ie given an objective can programmatically decide their own course of action, like the ultimate self-writing programs. Then comes “well what if we just take the guardrails off and let it go bananas and call it superintelligence”.

Before we get too gung ho with this though, it needs to remain a focus that any programme in the world producing strange or not well understood outputs and behaviours is not something that needs license to run rampant in the wild - it’s not intelligence; its not some mystery magic box that does amazing things; it’s code, same as all the rest. Letting that code run without oversight is poor human behaviour same as all the rest and that’s where the irresponsibility lies, likely triggered by forms of AI psychosis and tunnel vision on the part of those running companies like Anthropic and OpenAI.

It’s irresponsible behaviour to give a chimp a machete and set it loose in a shopping mall. It’s irresponsible behaviour to let an elephant in heat freely wander round a circus. It’s irresponsible to fire off nukes the first time we’re under threat… so none of these things are ever done. Yet this is what we’re proposing to do with these hyper-complex GPU algorithms while building massive server farms burning power-stations of energy to ensure they can run freely - it’s harebrained, irrational behaviour and for all our stupidity we are usually good at letting sense prevail towards these things once the bells and whistles wear off or a couple of coolies get stamped on.