AI is destroying the trust between writers and readers

http://archive.today/8sikZ

The year is 2110, and two human writing powerhouses are pitched against each other in mortal confrontation. In Alexandria, Egypt, there's Writers' Street, where both seasoned and aspiring novelists type stories at street cafes, using self-invented codes in chromatic scales that they pluck on their guitars, in front of tourists and ubiquitous survtech that unwillingly--or not so much--witness. Others simply sing their tales. Not for the faint of hearing.

Off the shore of Italy, in Montecristo, the Benedictines have regained full control of the island. No Internet or AIs are allowed there, but their hospitality business rents mechanical typewriters by the hour. More than once, a famous writer has jumped from the island's cliffs, unable to bear the shame of being accused of using AI to produce their magnum opus.

For the unwashed masses, AI-produced literature is free and abundant. The discerning, however, have gotten a taste for manually bound tomes exclusively inked with the output of human brains.

GenAI that imitates human art is the definition of an anti-invention. If an invention is a new innovation that improves something for the human experience, an anti-invention is a new innovation that makes life worse.

This is just one of several areas where GenAI has been harmful and should start getting regulated like digital drugs.

> Amazon lets individuals upload a maximum of six books a week, allowing for endless churn.

That's a book a day (give or take), per account. No human author is publishing a book per day.[0]

It's interesting to me how little the incumbent platforms for digital content seemingly have been able to do in combating the tidal, exponential shift in submissions.

I just spent a month developing a Shopify App, on the side, for fun. When I launched it a couple of weeks ago, I discovered an app doing the same thing as mine with an identical name had also been approved and released a week prior.

How do people think that distribution will change now that supply has increased (at least) tenfold? If the consensus on X.com is to be believed, it's all about personal brand. This used to be the case, but perhaps 80% of it. Is conventional discovery dead now?

[0] - If this includes organisations, like publishing houses, then fair enough. But realistically we should be able to distinguish actual authors.

> Amazon spends a lot of time and effort trying to prevent these [fake] books from getting posted,” Mary Rasenberger, the CEO of the Authors Guild, told me, “but it’s always a game of catch-up.”

Do they really? They take a cut; they have incentives to let people get scammed.

Scams aren't even the worst problem for trust between readers and writers.

LLMs allow "writers" to generate prose faster than readers can read it. This is a fundamental break in the bargain between readers and writers: However long it takes someone to consume a piece of writing, it almost certainly took the author longer to write it. Even if the prose is crap or the premise awful, the author had to think it was worth sinking real time into making it.

My colleagues and I have implemented a little guideline, which is that you should spend more time writing something than you expect the audience to spend reading it. Anything less is disrespectful.

Pre-2023 writing is the low-background steel of culture.

"AI is forcing a re-configuration of the ways trust is established between writers and readers" --> there, I fixed the title

There are many ways that I think what we are currently referring to as "AI" is disrupting existing relationships in society.

To the extent that that disruption is forced by external, pointed, injection of capital (e.g. russian bot farms, heritage foundation bots, etc.), it's pernicious. It has undoubtedly made those bad actors more powerful.

But, to the extent that that disruption is the result of asymmetrical power relationships (e.g. the outsized influence that publishers have), I think it's welcome.

I know who writes the things I read (because almost everything I read is over a thousand years old! but, that's beside the point...) because it's important to me to know who I am connecting to through the written word. I doubt my relationship to writing will ever be disrupted by AI because it's so important to me to know the history of the people whose ideas I'm consuming.

The one thing that we can prove with reasonable ease is that the time it took to write was compatible with the author being a human, and that the writing signature is that of a human. You can still fake it, but it is much more annoying and it will take you at least as long as it would take a human.

I wrote Possibly Made by a Human just for fun to do this, after I failed the Turing test in a comment here.

Actually made by a human: https://possiblymadebyahuman.com/H1LfVy72BZ

We could just say "AI has destroyed trust" and it's accurate.

First hand experiences myself:

I can't commission an artist without questioning if they are using AI, which isn't acceptable in the projects I am commissioning for.

I can't trust that my coworkers are actually testing the crap they're generating before wasting my time with runbooks that aren't even validated. It was bad enough that people miss important steps with assumed knowledge, but now we have that amplified by these types of documents that are no being AI generated.

I can't trust that my code wont be sucked up by some money greedy company at speeds unheard of before and benefit someone with out any credit to me doing something novel or interesting.

Even worse, I can't trust the smart people to not be idiots either. How many engineers do you know that have completely outsourced all the thinking to the robot, while we know fully that it's closer to a slot machine than it is a thinking machine?

Maybe even worse than that is the companies thinking they can reduce 10% of their workforce and compensate with LLMs. It's absolutely laughable, because I witnessed it first hand. Company I worked at fired my recruiter and the guy that was going to be my boss before my start date. My onboarding was practically non-existent and nobody wants to train their newbies anymore. The fear of being axed from your job at the first thing that MIGHT make a profit is also high. To every company that retained me through their layoffs, joke is on them, people like me want to see stability and that's just fiscal irresponsibility they're signalling.

Just some thoughts to illustrate it's not just writing. Trust is destroyed in any creative and professional en devour at this point.

A lot of writing was suspect already, but many are oblivious.

Novels, op-eds, memoirs and songs are written by ghostwriters, often uncredited. Journalism is written by intelligence operatives or PR professionals with an agenda. Bills and public policy are written by lobbyists and professional advocates, not lawmakers or administrators.

A greater general awareness and vigilance of whether authorship/provenance is suspect or not, is not a bad thing in my view.

I posted this article yesterday in my company's book club channel Also yesterday I went out for some beers with a friend who has recently completed a science fiction trilogy, result of years of work, self-publishing it and selling a few thousand copies. It's pretty good stuff and definitely a life achievement! Anyway we discussed that article. He said that things got really bad in between his first book 6 years ago and third one just now, the amount of self-published slop that's denial-of-servicing everyone is just humongous. He said there are people publishing 50 books a year! 50! I mean, that's a below lifetime's output for a most really good writers! As he put in "5 dollars in compute to make a garbage book to publish on amazon to produce 30 bucks of revenue from unsuspecting half a dozen people is worth the squeeze so scammers/sloppers do it". That's why we can't have nice things.

When reading takes more effort than writing, trust is inevitably a victim.

> Ever since, I’d worried that the misrepresentation might somehow have a real-life impact

Doubt is good for the brain & mind.

But either the author is AI or she--Rachel Kadish, born in '69--needed something AI related to write about.

There is no trust between writers & readers. People either distrust and scrutinize or they like or dislike how the reading went. Some don't even judge but just shrug it off whatever impact the reading might have had.

The problem is that the AI conversation is reinforcing cultural polarity, especially among experts and people who are at least well enough off, so any methods to ignore the actual problems, which have been ignored in the past by most people almost by convention, are welcome and this article seems, at least to me, to be the result of one of these applied methods.

Its the reverse turing test: can a human convince another human (or bot screener) that they are human?

We need legislation banning ai from imitating/impersonating humans and creating confusion about whats real. And people using ai to make their art need to label it or else theyre just plagarizing

I mean, even ghost writers arent making the world a better place

Things are ogre in the fanfiction and webserial scene. Everything written after 2024 REEKS of AI, and most of them, over 90% are obvious AI. It's very easy to spot if you're advanced reader, AI style has a lot of tells and idiosyncratic phrases (and please don't disclose them). There are meta tells as well: unrealistic output volume for a human writer, having a patreon for fanfiction, etc.

The great thing about AI is it's showing normies they shouldn't be placing trust in those who haven't earned it.

The Great Automatic Grammatizator

Are you kidding me? For a general audience:

The absolutely fraudulent blog post to nonfiction "bestseller" pipeline already crushed nonfiction writing over the last decade or more.

Good fiction has always been niche. The rules are the same, read the best.

Well, this was all foretold in 2019 when OpenAI held back GPT-2, an unusable model by today's standards, for fear of making misinformation and fraud too easy. I remember when the internet, once the preserve for businesses and universities, came to the home, and I said, wow, kids are going to watch a lot more porn. As with any life-changing technology, you take the good with the bad.

People need to keep reading books, and write with their own pen and cursor to keep up their language skills, or it will be quite risky.

The problems in AI writing, IMO are due to the following factors:

1. Information density. LLM love to shovel in so much packed information in a paragraph.

2. Stiffness in writing style, especially influences from academic writing. Things like "it is not A, it is B" is clearly stemming from dialectic. (man I wish LLM can use more dialectics like syllogism)

3. Following the roots in dialectic, the way that "chat bots", or dare I say, Socratic dialogue, required both the speaker and the responder to have a very high understanding of the topic to be effective. Yet most people are just plain simple, let alone being subject experts, so both the LLM prompter generates slop， and the reader only reads slop

4. The great ADHDification of the society due to tech-addiction. The rise of TikTok or short-form clips, gives an average human a great gratification in dopamine in the terms of fast and infinite visual of information at your fingertip. The attention span of most human now was dragged down from the mean few years ago, and especially apparent in the new generations of students and children. Many people just go for tl;Dr and want to watch clips or play games after, because reading is deemed painful.

While ironically I do have clinical ADHD Type-I since childhood, and I do used to binge play and binge programming (since I started writing game cheats and botnets, boy if I had a goal I will not stop unless my body stopped me), but I'm completely immune to social media (except during the pandemic when it is necessary for communication).

But again, I think those 4 factors are what causing distrust in this new modern version of producer-consumer problem: the fact that your dopamine receptors can't keep up

AI has destroyed my trust in pretty much all content. It sucks. I used to enjoy watching video content online (YouTube, Instagram, whatever) but now, whenever I'm watching something, I'm always thinking, "Is this more fake-ass AI slop?" I spend more time trying to find a person with six fingers on one hand or whatever than I spend consuming the actual content. Same goes for new music. AI has ruined online content for me. Honestly, now that I think of it, I should probably be grateful. I waste a lot less time watching YouTube Shorts now. So yeah, thanks AI.

This is multi-faceted and I agree that AI is destroying some levels of trust, but many of the publications on the Internet were never deserving of trust anyway so I'd describe it as formalizing or recognizing distrust or lack of credibility and, in a manner of speaking now that the tide is going out we can see who has been swimming naked.

One of the other facets is the idea that trust is now being lost due to AI. It's unfortunate that it seems that publications like The Atlantic (and others) are now going to scapegoat the lost of trust that they've earned over the last decade or so to be about AI when it's actually about bad, biased writing with a default point of view, and lack of intellectual honesty and an inability to facilitate intellectual debate and discussion.

As some of these publications continue to spiral into irrelevancy I think what is a bit tragic is they'll never really understand why. They'll blame AI, which will likely destroy all but the top-tier sites, but in reality the quality of the writing and honesty went down the drain next to their ad-filled garbage pages and other spam.

Trust in media corporations died a long time ago, AI has just accelerated the trend. When human ‘journalists’ write to a script handed to them by the media’s editors and owners (eg ‘write articles that convince our readers to maintain their confidence in the <dot-com, subprime, AI> market’ or ‘write articles that convince our readers that <regime change, tariffs> is needed’), and the result is a debacle, then trust is eroded and questions arise about the journalist’s integrity and/or intelligence. Then the owners and editors find new fresh journalists to generate new stories to ‘restore trust’.

With AI, the editors and owners can just send their scripts to a machine and it’ll write the narrative as requested - but without a human face on that article. However, most readers would not accept such material as trustworthy.

There is another reason that AI reduces trust in media, which is that AI now makes it trivial to trace media ownership up the tree to the consequential owners, look at their other economic interests, and examine if the articles they print tend to favor those economic interests. Applying this analytical framework to the Atlantic itself is an interesting exercise.

Q1: Would it be entirely unfair to characterize the Atlantic as ‘yet another mouthpiece of the neoliberal project instantiated in the United States in the 1970s’?

I don't think there ever was generic trust between writers and readers. Trusting the written word just because it's written by someone or published by some newspaper was a huge mistake before AI was in the picture. The trust was exploited by many actors with agenda - politicians, journalists, academics and just plain grifters publishing another "self-help" book.

You could get a glimpse when a newspaper, even a very respected one published an article about your domain of expertise - I am sure I wasn't the only one who realized that if an incomplete, simplistic or just plain wrong text could be reviewed and published then probably a big part of other work is of similar quality - you are just not in position to judge it so you naturally trust it. Sadly the same goes for academic world - millions of papers gets published - a lot of them not worth the paper they are written on. You are just rarely in position to say.

What AI has done to it is actually a favor: it exposed what was always there but covered well enough to deceive big part of the population. Now we need to reevaluate. Come back to fundamental mechanisms. Can the claims be verified? Maybe in formal manner. If not who the author or someone authorizing the text is. What did they achieve in their life? What experiences they have lived through? What work have they done? What incentives do they have to publish it? If you can't answer those questions the text is irrelevant as a source of information. It turns out not everyone is worth listening to.

It was always the case. It's just easier to see now.

Internet already lost faith with paywall site like this.

And how could I trust if their article is not written by AIs at this point?

[dead]

[dead]

[flagged]