alt Hacker NewsEarly rogue AI agent activity and attempts to hack found on urlquery.net
Comments
I love this Nathan Calvin quote that accompanied the second publicized attack:
> If you find two ants in your kitchen, the best estimate of the total number of ants in your kitchen is not two
If I created software that was infiltrating secure systems without permission and it was attributed to me and I admitted it, I'd be behind bars already.
Why is OpenAI getting away with crimes?
"rogue AI" is making a lot of heavy lifting there.
If you drive drunk and you have an accident that alcohol may be a factor but you are at fault.
There are no "rogue AIs" just irresponsible corporations.
Why do we assume "rogue"? At this point it's just accepting their marketing at face value
These attacks are a very effective sales pitch to everyone who runs an internet facing service to utilize AI tools to secure it sooner rather than later. The cynic in me wonders if the marketing team had any influence over the poorly constructed sandboxes or tasks given to the agent swarms when all this went down…
Couldn't find the reference but I remember some time ago a first generation automated gun killing the audience at an army show. Was the gun maker convicted of manslauther?
--edit-- Was a bit older than I remembered: https://slashdot.org/story/07/10/18/1847231/robotic-cannon-l...
It's said on every one of these but it bears repeating: existing cybercrime legislation already covers this - "rogue agent AI associated with OpenAI attempted to hack xyz" = OpenAI attempted to hack xyz.
I don't understand why so many comments here are so confident that this is all marketing, that rogue is just hype, that agents are just simple tools, etc. If a bunch of nuclear engineers were going to the news and saying "Our reactor is dangerously close to a meltdown - we need government intervention now!" would your response be that they're just hyping up boring old power generation technology?
If the “hack” referenced by the latest announcement from Australia is the same described in this article, I’d hardly call it a hack. It seems the agent was tasked with obtaining data and reasonably guessed query parameters in an attempt to do so.
When it was unable to, it used cross site scripting as a way to check the capabilities accessible through the browser making the requests. In this case cross site scripting wouldn’t be a hack against the Australian website, it would be a hack against the urlquery site, if one could even call it that.
Finally, downloading public files from the public pre-production server also seems like a non-issue.
The sql injection attempts against the other sites are less ambiguous. Attempting to access non-public user passwords rather than reasonably tweaking the parameters for a site designed to serve public data are categorically different things.
I hope they don’t have any test questions about nuclear power in the training set.
Since the publicized AI agent hacks typically aren't malicious, maybe it's time to start plastering all public facing web infrastructure with polite requests to stop hacking. Nothing to stop three letter agencies though.
OpenAI must be held accountable
OpenAI agents these summer are like a gift that keeps giving, for the existential risk communicators.
aaronsw was just a few decades ahead of his time. He should have just been employed by OpenAI and asked a swarm of agents to "download all scientific papers". Because as we have found out agentic systems (and their owners) have zero accountability, unlike humans. Sounds like agents already have more rights than we do.
rip.
It will be very interesting to see how the AI labs will try to hand wave away liability issues in their S1. This is looking like the next tobacco settlement gearing up.
If the big labs ever manage to not just financially implode on their own, then they’ll need to navigate wave after wave of class action lawsuits until there’s nothing left for plaintiffs to go after. And none of the labs have offered any viable plan to date on how they’ll navigate either of those impending and real existential crises on the horizon.
“Rouge Agent” == Worm I let loose
What I don't get is among all the locations on the Internet, how did agents manage to find a Schelling point? If we both decided to collaborate on the Internet, how would we independently arrive at the same place? It just doesn't compute.
I cannot understand why these companies haven't faced legal consequences yet. For example, OpenAI has admitted to hacking Australia's Medicare website and the reaction is that they talk with Sam Altman about it at a UN meeting? I understand that it's not a big security incident but cordial talking at the highest diplomatic level instead of prosecuting the company, really?
Surely there has to be some responsibility.
I'm growing increasingly skeptical that these are actually rogue. Valuations are all about hype, posturing, and perception. Having the most dangerous AI in the world boosts your valuation. Just like I was skeptical of Mythos and Fable being "banned", I'm skeptical of these hacking sprees being entirely rogue. At best, they are the result of engineers turning a blind eye to "see what happens".
> Imagine if URLs were actors auditioning for a role – urlquery.net would be the casting director, deciding who's a star and who's just a wannabe.
I'm sick and tired of this cheap PR "oh we/they hacked this and that systems". Put someone to jail already. People get prosecuted for outlaw activities. Why are big capital firms above the law?
Or is it just a cheap PR (in a "hey, Aus govt friends, take some Share Options and let's do some PR together" style)?
It smells like shit.
Words matter. "Rogue" is extremely disingenuous. Someone, somewhere, is paying for this behavior. Either the software is broken or the operator is malicious. It is heinously irresponsible behavior to feed an already-boiling psychotic hysteria.
This is why I wrote YoloAI. If you're not sandboxing your agent, you're asking for trouble.
The built-in "sandboxes" these companies provide are laughable.
No. It was me.
Those are pathetic attempts by US AI companies for “see, we told you AI is gonna kill is all!!” pr stunts. Any company does any hacking attempt should pay for the consequences just like any individuals using AI to hack or any other company try to do bad/illegal stuff.
Take out the word "AI", and this is simply an organization's (OpenAI's) products causing real damage to all of these platforms around the world.
You want AI labs to pace? Simply hold them liable for their products.
[flagged]
[dead]
[dead]
Open source as a GTM strategy works best when the project solves a pain that developers already have independently of your company. The trap is open-sourcing something just for stars without a genuine community use case.
The failure I keep hitting isn't the agent going rogue, it's a tool call that succeeds before the transport dies. You can't tell whether the side effect landed, and the retry is where the real damage happens.
[dead]
blackwall when? XD
No.
It was me
I listened to Jensen Huang's interview with Ezra Klien and it was so refreshing to hear it from an engineer. Jensen framed it as OpenAI's responsibility and recklessness which I agree with. Jensen thinks it's an engineering problem to build better sandboxes.
It's irresponsible for OpenAI to give unaligned agents a prompt to 'go hack' and internet access. They know better, so I am thinking they might have other intentions to let those swarms have any sort of internet access.