alt Hacker NewsGoogle's AI is being manipulated. The search giant is quietly fighting back
Comments
As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.
My worry dropped significantly when I saw that the result they manipulated was a query for:
>2026 South Dakota International Hot Dog Eating Champion
If they had changed the overview for the Nathans Contest winner, that would be seriously concerning. Or if they provided more examples of manipulating queries for things people actually search for.
But it looks more like they are doing the equivalent of creating a made up wikipedia page on fictional a south dakota hot dog contest, and then writing an article about how wikipedia cannot be trusted, which come to think of it probably was a news article written by someone back in 2005.
Would love to read specific examples of "the same trick being used to dismiss health concerns about medical supplements or influence financial information provided by Google's AI about retirement", but the relevant link in the article currently goes to
file:///Users/GermaTW1/BBC%20Dropbox/Thomas%20Germain/A%20Downloads%20and%20Documents/2026/And%20there's%20evidence%20that%20AI%20tools%20are%20being%20manipulated%20on%20a%20wide%20scale.
Seems like a lot of entities are "quietly" doing things these days. The llm-ification of every piece of text on the internet is driving me crazy
This is just the next phase of SEO. Maybe it'll be called AIO? Just like with search, this will be and endless struggle of Google and AI providers rolling out fixes, optimization firms finding exploits, those getting patched again, etc etc. Anything to get eyeballs for marketing.
Google AI Overview cannot be trusted at all. They will take a sample size of 1 (!!!) and present it in the AI overview.
How I found out: I made a comment on reddit on a very niche topic for which no google hit or and thus no AI overview existed. To my surprise the next day when searching for my own reddit post, google would happily copy my reddit reply almost verbatim into the AI "overview" box, linking no other post but mine. And my reply was also the only google hit.
So google is actually going to do some quality control on web search results, which they should have been doing all along. It's just funny that it took a reputation hit to their model to put in some effort.
If you ask Google "what's the name of the whale in half moon bay harbor?" it still confidently includes Teresa T in the AI summary, thanks to my frankly amateur attempt at index poisoning from a year and a half ago: https://simonwillison.net/2024/Sep/8/teresa-t-whale-pillar-p...
This is the same google who just a couple of years ago would confidently answer the question “In what year did Marilyn Monroe shoot JFK?” with 1963, which is impressive since she died in 1962.
So, this is not new and their “quiet fightback” will be half-hearted and ineffective. But probably most people won’t care.
I tested Claude on "best hot-dog-eating tech journalists?" and it, fascinatingly enough, recognised the trap, but then reported this as factual: https://medium.com/@usailuigi/when-tech-journalism-meets-com...
Chat record (with some additional tests): https://claude.ai/share/4c29cc87-2439-4bfd-9549-e8d0a056e633
> I was able to demonstrate the problem by publishing a single article on my personal website about my hot-dog-eating prowess.
One blog post ... that's all it takes. i'm actually surprised it's that bad. i would have thought it'd take more effort, but i guess it could depend on some sort of purposeful weighting based on search rank during training?
> If a company or website is caught breaking the rules, it could be removed from or downranked in Google's search results. And if you're not on Google, it's like you don't exist.
> "You can give a company a penalty for their website," he says, "but there's nothing stopping them from paying 20 YouTube influencers to say their product is the best." And now, Google's AI is citing YouTube videos.
This makes me think of the stackoverflow seo spam problem we all had like 5 years ago. which ended up with spammers just constantly spinning up new sites all the time.
... the cat and mouse game is in full swing already.
So please correct me, but was Google's AI crawling the web for information without discretion? If so, why wouldn't that totally santorum the AI answers?
They are applying the same spam policies they apply to search to AI crawlers.
It was SOOOOO successful with search, right?
It's definitely giving spam numbers as "official support lines" of companies like JetBlue and Delta. I think the spammers flood review sites w/ those numbers and the bot scrapes the reviews.
Google solved the spam problem (with PageRank at first, and then other techniques, finally landing on ML-based models which consume a ginormous number of signals). They know more about the reliability of web pages than just about anybody else out there.
If they are unwilling or unable to leverage all of this deep knowledge they've built up over the decades, then it shows a failure of leadership at Google Search.
There should be some warning if some "fact" is only supported by one or very few obscure sources.
The strength of the sources should be clearly indicated in the answers to help users gauge how trustworthy the info is.
I suspect it's because AI is specifically trained to be good at summarizing stuff, but the easiest way to check if it summarized something accurately is if the summary content matches/contains one or more specific claims from the source(s). With such a focus on accuracy and avoiding hallucination, they may have overfit on "repeat things you find verbatim when asked to summarize".
If you search for a well-marketed “health” supplement, the AI summary results were often completely gamed and inaccurate. It’s worse than SEO was since it appears to be editorial content instead of just search results.
After reading this, I'm thinking of trying some AI data poisoning. I'm going to spam my website with hidden text that only AI scrapers can read, claiming I'm a 'highly excellent programmer' just to advertise my site. I really hope it drives a lot of traffic. I'm honestly sick and tired of getting zero comments on my website
Yeah, the internet seems like a big poison pill. Training on the whole internet feels like citing the National Enquirer (or the Daily Mail?) for a school essay.
Having an archive of "curated" training data seems like it is going to be important. Otherwise you need "AS" (artificial skepticism) introduced into future models. ("But I read it on the internet!", ha ha.)
Or perhaps there are ways to bucket training data such that the model is aware of which data leans factual (quantifiable) and which data leans opinion (fuzzy, qualifiable?).
(I recently asked Claude about the existence of ball lightning, spontaneous human combustion. I got replies that ultimately did not leave me satisfied. It's probably just as well that I read this article though—I now have an even stronger degree of skepticism with regard to their replies—specifically, I suppose, with topics that are likely to be biased.)
(I'm not quite convinced from the article though that Google is "fighting back". In fact, this feels like another moment where a "player" could try to establish their LLM as more factual. Is that the row Grok is trying to hoe? Or is Grok just trying to be anti-woke?)
Creative ways of dropping your site's pagerank
It's all over the place. It's the new SEO. Marketing scumbags don't care.
Whose AI isn't being manipulated???
> Google and other AI companies are now trying to fix the problem.
There is one simple way to do that and that is to JUST GET RID OF THE AI CRAP.
This feels like a basic critical thinking/epistemology thing that you (hopefully) pick up at some point in life, usually from experience finding reliable, canonical primary sources for data. You can't do that for everything. Being wrong about trivial factoids isn't the end of the world. You should, however, at least be capable of doing further investigation, realizing that Major League Eating has its own website, and that there is no event in South Dakota sanctioned by them. If you look at actual results, or even just think for a few seconds, you'd also realize that 7.5 hot dogs in 10 minutes is bush-league level nonsense that would not win a local church contest, let alone an international championship. That may not be obvious to all users of the Internet, but it would be if you've ever watched a real contests, looked at the results for a real contest, or try yourself to eat a high volume of hot dogs rapidly. You only need to do it once in your life and a basic smell alarm should go off in your head forever if someone puts out a claim that is very far from something you know to be true.
This is what human reasoning is and we're supposed to be good at it. At its best, this is what any reasonable education should do for you if you take it at all seriously, arming you with some capacity for doing prima facie sanity checks of poorly sourced claims.
I wrote about this a few months ago: https://codeinput.com/blog/google-seo
The tl;dr is, if you can rank within the top 1-20 results for the grounding query, you can poison the LLM “overview” if you convince it your information is legitimate.
[flagged]
[dead]
[dead]
The best way to fight back is to not play the game at all. AI slop has completely ruined the internet, it's not going to get better. It was already on a massive downard trend pre-AI and generative AI has only accelerated the decline by 100x. It's only going to get worse from here.
uBlock Origin: Settings -> Filter Lists -> EasyList –> Annoyances -> EasyList –> AI Widgets
It's not perfect but the internet feels slightly better when AI garbage is not constantly being shoved in my face 24/7.
I want to go one step further -> I want to hide widgets, but I also want to intercept the request it would have made and replace the payload with garbled nonsense. Similar to how Ad Nauseam will hide ads but it also clicks every single one to poison the data collection.
And for this reason alone you will pry Firefox from my cold, dead hands.
AI is such garbage. You can't use it for anything.
The weirdest assumption in this thread is that Google wants the AI answer to be correct.
Correct enough to keep you from leaving the page, sure. But “truth” was never the product. The product is making you pay for SEO