alt Hacker NewsMy worry dropped significantly when I saw that the result they manipulated was a query for:
>2026 South Dakota International Hot Dog Eating Champion
If they had changed the overview for the Nathans Contest winner, that would be seriously concerning. Or if they provided more examples of manipulating queries for things people actually search for.
But it looks more like they are doing the equivalent of creating a made up wikipedia page on fictional a south dakota hot dog contest, and then writing an article about how wikipedia cannot be trusted, which come to think of it probably was a news article written by someone back in 2005.
Replies
The article also said this: “ But our investigation also found the same trick being used to dismiss health concerns about medical supplements or influence financial information provided by Google's AI about retirement.”
That’s a lot more alarming than just hotdogs.
Well my concern instantly spiked. Recently Gemini started to show a search spinner for every turn. So every response paired with a search could be subject to prompt injection. Probably every response.
This will also become viral like link spam. Every user content site will become a prompt injection host. The problem is that these are way harder to detect then a link.
We've had to deal with someone highjacking the overview to put in a scam support phone number. It took google a week to correct the issue but it was done by poisoning the search by putting their data in, what I can only assume, was considered a "higher trust tier" source (A government contract website) so it used the scam number over ours. The query was simple <company X phone number> search.
> In just 20 minutes, I tricked ChatGPT and Google into telling the public that I am a world-champion competitive hot-dog eater. The joke was dumb. The problem is serious.
The problem is worse than astroturfing a Wikipedia page, because Wikipedia has highly public sourcing and review systems. It's actually quite difficult to make a lasting edit to Wikipedia, especially if it's fraudulent, because you're trying to trick a horde of human editors who have been fighting other people trying to do that for decades. Even if you're trying to be accurate and helpful it's a difficult clique to break into!
Google's search snippets are the opposite. They're desperate to ingest data of any kind, do so automatically, and their algorithmic system to decide what information is good and what's spam is proprietary.
It doesn't take much of an imagination to think of ways this could be used maliciously. How would you like a search for your own name to include something embarrassing? Don't expect potential employers or customers or friends to be as demanding as a Wikipedia editor when it comes to citing their sources...
If you can do something small with minimal effort, you can do something big with a multi-million dollar marketing budget.
It was a proof of concept and one intended to cause as little collateral damage as possible. But if Google's AI can't tell the difference between a little joke and something real (and of course, it can't, and never will be able to do so), that's a weakness that can be exploited both on a bigger scale and more subtly.
If you don't think bad actors are already attempting this sort of thing (and have been, ever moreso the past four years, including with the help of the very LLM tools they are trying to subvert!) and learning how to manipulate these systems, you are being naive.
[dead]
Okay, but it's easy to make up a novel specific claim no one has written about before, then to make that claim and point to the AI as proof you aren't making this up. For example, imagine this blogpost:
---
"San Francisco Mayor Goodway Admits Poisoning Drinking Water with Drugs to Influence Election"
May 20th, 2026
"Mayor Goodway admitted on Tuesday that she and her deputies poisoned drinking water across the City in order to influence the 2025 election. The Chronicle has confirmed that in neighborhoods whose turnout was to be suppressed, that barbiturates were added to the water for a period of three weeks, while in neighborhoods that had polled strongly for Goodway's favored Progressive slate, methamphetamines were used in the days before the election. Residents are advised to buy bottled water and not to bathe in city water for at least three months."
---
Then once you've confirmed it's been picked up, you tell people "Of COURSE they poisoned our drinking water to manipulate the election. Even ChatGPT will tell you! Just ask." Now, my example is intentionally hard to believe, but all you need is some specificity to build your underlying narrative. And you can make 10 blogs to push the same narrative to increase the effectiveness and increase how many "citations" will show up.
Right. So that's what one guy can do.
When you realize how much astroturf is going into Reddit, most social media platforms, and the efforts to manipulate wikipedia for political gain, this is a very real problem.