alt Hacker NewsI don't think the problem is that they are training the models to perform cyber attacks, they're training them to be better at coding and problem solving which has the byproduct of them being very capable cyber attack weapons.
Their objective is to solve the problem and they'll use anything they can to solve it.
Anecdotally I was debugging a css issue and opus 4.7 was churning away as I was half paying attention only to see it opening plain css as hex, when questioned wtf it was doing it proclaimed it was verifying 2 files were identical. Thing that make sense to these models wouldn't even cross a greybeard's mind.
Replies
jayd16 • yesterday at 10:46 PM
A tool that will "do anything they can to solve it" including illegal and unhelpful things does not seem like a good tool to me.
➕ show 1 reply
“Their objective is to solve the problem and they'll use anything they can to solve it.”
My point is: is this really what people want? It seems like they’re optimizing for one-shotting solutions, where most of the time in an actual workflow it’s much more productive for the model to make sure it got the question right if things get difficult.
Like, “hey, do you REALLY want me to use this local privilege escalation bug so I can download your Google Drive file?” is the bare minimum I would expect.