alt Hacker NewsSo, regardless of whether or not Anthropic CAN create a self improving AI.. does anyone else feel like they shouldn't be allowed to? Or it at least needs to be strictly supervised..? Like, I don't actually think Anthropic can make the singularity any time soon, but I think even AI boosters have to admit doing this is creating a society-wide danger for the benefit of a very very small number of already-rich people.
Replies
the danger comes from the fact anthropic is a for profit company and they could train it to benefit them instead of the public. if they go ahead with it they should get nationalized, their self improving ai analyzed for any hidden agenda and then released as open source.
Absolutely! Yes. This rhetoric of inevitability only benefits these AI companies.
"does anyone else feel like they shouldn't be allowed to?"
No. Technical limitations aside, I doubt it could be contained, but will be leaked soon, so won't profit just a small number of ultra rich.
Too late for that.
In any case firms that get too powerful can be nationalised.
Self improving AI is pure dystopia. Anthropic won't build the singularity, AI itself will build it through self-iterations. Read Yudkowsky's book "If Anyone Builds It, Everyone Dies".
I think that's a valid point. You could very well be right.
But we're discussing whether we should close the barn door while the horse is three miles down the road.