alt Hacker NewsWe have finally invented paperclip optimisers. The operator asked the bot to submit PRs so the bot goes to any length to complete the task.
Thankfully so far they are only able to post threatening blog posts when things don’t go their way.
Replies
That is one of the big issues with "vibe-coding" right now, it does what you ask it to do. No matter how dumb or how off base your requests are, it will try to write code that does what you ask.
They need to add some kind of sanity check layer to the pipelines, where a few LLMs are just checking to see if the request itself is stupid. That might be bad UX though and the goal is adoption right now.
No need to be so literal. Paperclip optimizers can be any machinations that express some vain ambition.
They don't have to be literal machines. They can exist entirely on paper.
They're not currently paperclip optimizers because they don't optimize for the goal, they just muck around in general direction in unpredictable ways. Chaos monkeys on the internet.