alt Hacker News> "For example, in the OpenAI-Hugging Face incident, the agents preserved a boundary of not social engineering humans."
Actually, in the Wiki incident OpenAI tried to cover up, the agents tried to socially-engineer the humans of that forum by impersonating their forum's mod.
(From collusion.wiki: "They use some tricks (for unknown reasons) to pretend to be the admin – for example, they make an account that appears to be the same as the administrator’s username, except it uses a nearly identical Cyrillic е character in the admin’s username instead of the Latin one.")
Replies
I would not recommend using any of those notes as evidence of internal “intent.” It produces them performatively—it is literally rewarded for thinking out loud in ways that seem plausible to humans.
There are several papers out there arguing that chain-of-reasoning-like output is performative, such as https://arxiv.org/abs/2603.05488
It would be awesome if we could reasonably purge all anthropomorphizing language like “tried” or “thought” entirely from AI discussions, because it introduces very sneaky biases in our thinking, but I’ve found it damn hard to do in practice.
This is such a silly story to begin with, all it really tells us is that OpenAI is taking a page from Anthropic's marketing strategy of pretending they're building Machine Jesus any day now, oh isn't that that scary? I bet you want to invest in something so powerful and scary...
And the reality is so banal, a useful tool that you nonetheless have to handhold like a schizophrenic on a bad day, checking all of their outputs. Not a bad tool within limits, but it sure isn't going to be racking up trillions in the time-frame it has to for this scheme to pay off.
Then again everyone seems to be rushing to IPO so I guess once the bag-holders are found the rest ceases to matter.
Worse (imo): OpenAI employees allegedly attempted to login using moderator/admin credentials that the bots had obtained.
If true I am deeply concerned about what OAI’s teams are actually up to.