Zombie stats: Where the numbers AI quotes come from

To me the dream of AI is: an agent can do the drudge work of tracking every statement down to the supporting reference, documentation and ultimately the work involved in verifying.

It is NOT being used for this, instead we want faster “answers” that avoid due diligence. Same as always but lazier.

I can't decide if the report's writing ("One number, followed" & "Quoted by all five. Sourced by none.") is being lazy or ironic?

The same thing happened to me while building quote extraction agents. AI would often extract figures that are not stated anywhere in the quote document. This has reduced with models getting better, but we still ask it to cite the exact line it got the figure from.

I know Perplexity isn't often talked about anymore but their design of every sentence needing to be able to be referenced back to an actual source (like an academic peer reviewed paper) seems like a super power for trust.

A full 40% of AI stats actually have a specific study behind them? Knowing how the human baseline is that 73.6%* of all statistics are made up on the spot, this is really encouraging.

*That particular stat was of course made up, but not by me.

Just an thinly veiled ad for something called trakkr. Not to mention that none of these "stats" actually matter aside from being the type of thing that a consultant would put in a pitch deck, at which point their accuracy doesn't matter anyways

And this doesn't even touch on the fact that a significant fraction of the published, peer-reviewed, authoritative sources ultimately turn out to be wrong[1]. Science is hard. Data collection and cleaning is hard. Mistakes are easy to make.

[1] https://journals.plos.org/plosmedicine/article?id=10.1371/jo...

This article is complete slop and not worth looking at, but it's an issue worth discussing.

One of the original php2shell announcement posts (which was clearly AI written) claimed that 70% of websites use PHP. This number has now been repeated by every single article about php2shell, including in the most trusted tech news outlets.

I think it could have been approximately true in 2008, but not in 2026. The number is too round anyway. Trying to search for a source to the claim is now impossible.

If the basis of a reliable fact is consensus, then every AI model should now consider this fact to be true. In reality consensus is not useful of course, offhand facts are frequently generated by citogenesis. An article mentioning a stat offhand is not a source. A source should take some amount of epistemic responsibility for a claim.

There is a human/user element here as well - will the users check the citations, if they are provided inline? I recall in the early ChatGPT web search and Perplexity days, most of the citations didn't work (made up, or 404). You get a false sense of security by having a "[1]" next to the stat or a claim. Maybe labs companies should have some additional quality ranking/checks on these, if it's in their interest of course.

Well, 78% of all statistics humans mention are made up as well, so.. :)

Never trust a statistic you haven't sneaked into the training corpus yourself.

While it's interesting to see which models performed better I wish they did a control. Gave the question to x number of people and had them provide an answer.

bad (or even made up) statistics have been floating around meat space since forever.

It just goes to show that when AIs are relied upon to recall information, they may hallucinate.

It lends credence to the theory that a smaller model without the ghost of information past, could do better with ability to search the web, although this is heavily reliant on the effectiveness of the search engine itself.

Good findings, but the findings are so AI-generated and hard to read… Feels like I am being thrown numbers without a coherent story, and even worse, with beautiful animations which only serves as distraction… This looks very similar to the diary entries my LLM writes when working on a coding project.

I don't trust LLMs to give me numbers like this, and it's not just that it might make a mistake on the number or it might have garbage in its context - it also might just not understand the mildest nuance in its context or apply any common sense.

I recently asked what percentage of ice cream is air. It told me up to 100%. When I followed the sources, as I always do and not just with silly answers like that, the number in its context was indeed up to 100%. There's an industry term called overrun, measured in percentages. But 100% overrun means it's 50% air.

So why would I trust any of these numbers in this obviously LLM-generated and probably LLM-researched article?

I guess it is slightly clever, if you feel like slipping out posts, to make them anti-AI. Really blunts the effectiveness of criticism of putting slop out when the slop agrees directionally with the criticism.

Ah yes. The AI slop blogpost calling out AI slop statistics. We must go deeper

Better to be on the 40% or the 60%? Seems like both sides will bite you in the ass to an extent

Google's AI is extremely prone to making up random shit that doesn't exist. Swedish is an agglutinative language (like German) so you can join multiple words together to form new words: tomat (tomato) + jos (juice) => tomatjos. Most conjoined words are nonsense, but Google will still happily "explain" what they mean. Some examples:

tangentbordskanin (keyboard bunny): keycaps for mechanical keyboards with bunny images on them

indianbokföring (Indian accounting): simplified accounting outside of legal norms

bokföringsindian (Indian accountant): informal, older term for an auditor

potatisbensinskämt (potatoe petrol joke): a type of classic absurd riddles or word plays

nagelfis (nail fart): bulging pain in fingers after having been outside in extreme cold without gloves

And it regularly lies about local stuff, e.g., it says that coffee shop A has lower prices than coffee shop B when it's the opposite in reality.

An apple a day keeps the doctor away and Santa wears red.

The determination with which AI and fitness trackers deliver numbers, plots, and statistics is really seductive.

AI, turning the joke that half of all statistics are made up into an accurate statement.

So no different, than humans repeating facts we hear every day as 1 in 5 dentists tells us

That's why a transparent attribution graph is paramount for trust-worthy AI. Otherwise paid disinformation campaigns, as already spewed out by for-hire private-spy agencies (NSO and similar) and 'LLM-SEO' in general, to poison LLMs will deteriorate LLM search to become as enshitiffied as the 'classic google/bing/etc ad-driven' search beforehand.

when people say "there should be no AI taught in schools ever. prompting? what skill is there in talking to a liar bot?" this is where they are dead wrong. Being correctly critical of LLM output is a huge skill and news flash it applies to people and human-written things too.

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[dead]

[dead]

[flagged]