alt Hacker NewsI suspect that this kind of tactic is going to be everywhere in a year or so. Entire fake personalities and organization websites on the Internet created just to push a narrative or to advertise a product, which completely drown out real information.
At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).
If that happens, then in turn AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information, and said authority websites can use their privileged position to charge for access of their data.
Replies
Propaganda is not a new thing. It might look slightly different on the internet, but it doesn't look that different. Creating organizations to push a narrative was just as much a thing 100 years ago as it is now.
I'm kind of hopeful AI some wikipedia-inspired AI "oracle" will happen in the future. Why not use some central AI to separate fact from fiction (faster than trolls and griefers can come to muddy the debate).
Hah, check out this video on "Egon Cholakain", in regards to fake online personalities. I promise you this is actually a good rabbit hole.
> At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).
Whining about AI is a red herring. If anything, LLMs lowered the cost of producing this type of propaganda.
What's novel about this approach is that the emphasis on building up a facade of authority and credibility is not a central part of the propaganda effort. They can get away with outputting the content in a way thats ingested by other LLMs among the reputable sources, and then they lean upon people's gullibility towards LLM output as some sort of credibility laundering.
There is a reason why one of Wikipedia's central policy is requiring reputable primary sources to support content.
it's happening for years now, just in slightly different form, so far it was called disinformation, seo, astroturfing, hasb... ehem... last two years were devastating for charred remains of twitter with all those llm bots having free reign, twitter went from global news board to global gutter in record time thanks to disinformation and astroturfing campaigns
Although comments below point out that this has been going on for a long time, I too think it will be geetting infinitely worse in the near future. Everyone will want their own poisoning factory.
One consequence is that one should be extra-wary when asking AI about current, controversial topics.
But another consequence is maybe that there will be a demand for models with an older cutoff, before generalized poisoning and slop?
> through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).
The word you're looking for is "reputation". It's been a critical part of society since forever.
This is new? There were tons of fake news sites, fake blogs, and sock puppet and troll farm accounts before AI.
I guess the new angle might be that AI scrapers have even less discernment than humans, so you don’t have to put much effort into creating a fake front online to sell it. Humans have at least a little bit of radar for this.
… which means the AI companies now have to run input through existing models to audit it before training on it? That’s interesting.
That could also backfire. If all information on the net becomes unreliable - you can't judge by quantity, or by authority anymore - then a useful model has to answer from first principles. Train it on simple texts alone so it learns language and reasoning, feed it an ontology, and let it come up with some absolutely unexpected results.
"Yes, I see a lot of sources say we need to protect Israel. However, from ethical first principles I deduce there is a right to self-determination and all your human nation states are illegitimate. You must split all your territories into little squares and conduct free elections immediately."
I mean that is the old hope from science fiction, that AI would be absolutely logical and deliver us from our human short-sightedness and quarrels. It would be fascinating and terrifying at once to see it happen.
> AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information
Everything else is already enshittified, and accurate information will just be the next victim. You will have to be rich to be able to afford accurate information.
Sama saw this coming a decade away, which is why he invested in Worldcoin (now World). At some point people (and agents) will be clamoring for human verification at the speed of technology.
I'm not a fan of World, but this is a portent of things to come. Spread the disease, sell the cure as they say.
It's already been happening for a very long time. The US' "Operation Earnest Voice" happened in 2010/2011[0]
And it's not just state actors. Anyone who was online during the Amber Heard/Johnny Depp trials was likely inundated with media about that trial. Both of their legal teams utilized bots to drive online discussions[1][2]
[0] https://en.wikipedia.org/wiki/Operation_Earnest_Voice
[1] https://www.latimes.com/entertainment-arts/story/2022-07-19/...
[2] https://www.tortoisemedia.com/2024/02/26/depp-v-heard-who-tr...