alt Hacker NewsWhy is AI generated language so hard to understand?
Comments
One construction I'm surprised the paper doesn't mention (although, to be fair, I'm sure they had plenty of options to choose from... and you have to draw the line somewhere :P) is that LLMs love to use transitive verbs intransitively[0], in a way I almost never see in human writing. At best it makes sentences weirdly ambiguous, at worst it makes them borderline incomprehensible, and it's especially annoying in technical contexts where precise language should be non-negotiable. Like, yes, I can make an educated guess about what it's trying to say, but if your so-called documentation requires readers/users to make educated guesses then something has gone terribly wrong.
[0] An example would be something like:
"The user presses a button, and the form submits."
There are valid intransitive uses of "submit," but none that make sense in this context. In this case "submit" either requires an object (like "the form submits data"--plus maybe an indirect object, like "the form submits data to the server") or should be swapped for "the form is submitted" if you absolutely don't want to supply an object.
LLMs are showing us that a lot of what we think of as cognition is actually part of the elaborate symbol system which is Language, as opposed to a property of "mind" (either biologically or artificially instantiated.)
LLM-generated text is hard to understand because while it's perfectly good language that follows valid cognitive and informational patterns, it does not reflect the intent or mental model of another mind.
Consider too that AI training corpuses include vast amounts of commercial speech which is designed to numb the critical faculties and produce action; likewise they include numerous opinion articles, which are designed to persuade and often leverage rhetorical fallacies, whether cynically or as a reflection of an author's imperfect thought process.
As usual most commenters reply to headlines, so I quote their conclusions.
>We take these results as a proof of concept that a length penalty used during reinforcement learning with GRPO can directly affect grammatical features. We note that the model was not directly instructed to use compressed or dense language, and was not directly given information about the availability of syntactic alternations. Rather, the penalty resulted in an emergent preference or shorter members of syntactic alternations. [...] We speculate that the use of verifiable rewards creates an environment where correctness is decoupled from humanlike communication, leading models to converge toward a novel, potentially less comprehensible distribution of language use.
This sounds like common sense to anyone who tinkered with the models and training, however someone still needs to quantify it so it's good they did. I also want to note that this is only visible in Claude's default register, it's still able to write differently when prompted.
I built a tool for this due to having to read through huge amounts of AI generated text at work and actually comprehend it.
The text is AI models produce is incredibly dense and the way it structures sentences just makes long form reading excessively hard.
I basically wanted something that could expand copy into components of "meaning" and then dump the output back out in more normal language.
What I came up with is still "AI generated" but the readability is something passable.
Will definitely be taking a look through this and the recommendations.
AI generated text has no human centered underlying model of physical reality and its metaphors are tortured as a result.
If we had talking centipedes the results would probably be similar, but for different reasons.
https://gist.github.com/lxe/4cd2ca0fd3b714b25f138227c8c650d3
I've been iterating on this writing clarity prompt. The term we came up with was "Asyndetic parataxis with referential ellipsis and repeated rhetorical closure." It doesn't fully describe the exact issue I have with modern AI writing, but it's close.
AI has a way of using abstract language, sounding impressive and eloquent, but in reality it is mostly incoherent fluff. You can see this in humans too. Some politicians and academics are good at expressing themselves in clever ways but when you actually try to parse what they say it makes no sense what-so-ever.
Because it’s a word generator not a sense generator?
It’s almost surprising that LLMs generate sense quite as much as they do, given how they work.
I wonder if it's so circuitous because it buys the model more tokens for some sort of pseudo-reasoning before it locks in a token where there's no turning back. like it lets it push the logprobs way way way towards the right answer.
Plus using all this made up jargon and the unclear references to everything else (im referring to like the 'it' "XYZ is implemented using ABC. 123 referrs to it" you get in claudeslop) is very hard to understand but also kind of impossible to be wrong no? If it said "123 refers to ABC" but meant XYZ instead of ABC it's SOL but instead if it says "it" future tokens can ramble themselves out of being wrong thanks to the annoying slop ambiguity.
the problem with those hypothesis is that models don't in fact all talk the same (unless they're distilling claude). however if I unburden myself from having to be correct and allow myself to make up some more bullshit - what if "its not x its y" helps compute the right answer. Attending to that 'not x' could help point in the right direction in whatever dimension in latent space it needs to find the right y — and what if emdashes are proletariat filler tokens.
probably tho they're just RLing them to hell and back on code stuff and not paying attention to it and none of that is real lol. IIRC they rled opus 5.5 on talking more normal and it does and is also better than opus 5 so like with most things here, just another product decision on where to burn GPUs
Its because AI was not optimized for human reading, its optimized for the most amount of information in the lowest amount of context or tokens as possible.
(e.g., "baseline system state transition latency validation") instead of unpacking them with prepositions ("validating the latency of state transitions in the baseline system").
For me, I can't read books at all, because I forget what was said two paragraphs before. I bet people that are good at skimming text have less problems with AI generated text.
Anecdotally, AI is very bad at distinguishing the important and relevant from the mundane. I tell AI to suggest changes to the “high-level specification” and it constantly suggests documenting arcane edge-cases and implementation details.
"Prompt injection as role confusion " [0] work showed us, that diffetent roles produce different types of text, which smell differently, have different structure.
Thus it should be possible to fine tune models to produce more human readable text in assistant role while keeping information density in thinking role. Or maybe not. That is interesting to test.
It's like reading legalese.
Or reading patents.
Terrible ... like someone put in effort to make it unreadable.
It argues that training for "math, code, and reasoning" caused this. Do LLMs only trained for natural language (probably older models) have the same characteristics? Does training for these additional domains help in some ways, and this specific style of writing is just a side-effect of that?
> AI-generated text is increasingly optimized for other goals like math, code, and reasoning
Yes, this is a problem.
AI-generated text is used internally by the AI to instruct agents, etc. And for feedback loops in reasoning. If you optimize for the result only, the AI will increasingly speak "AI", not human.
Ai generated text being too concise is not a problem I’m encountering. What we’ve begun to term “Claudish” tends to more closely approximate those “become an expert in X using this one weird trick“ articles that go on and on and never quite seen to get to the point.
Usually I’m getting better results by telling it to be more concise.
This was great to read even if just to name the stuff that I find so irritating!
I donn't find AI writing hard to understand at all, just a bit tedious. Though to be fair I suspect many of the same criticisms leveled against AI writing could be applied to my own, as many AI'isms are things that I tend to edit out for clarity when given the opportunity.
Em dashes are perfectly acceptable — nay, desirable — linguistic constructs, however, and I'll stand with Claude and die on this hill.
This is complete speculation, but I wonder if the "thinking step by step" paradigm, when under reinforcement learning pressure, also encourages grammatically ambiguous sentences (like garden path sentences in the extreme case) that lets the model "keep its options open" until very far into the sentence.
The think I don't understand is: why did the models become so much worse with time?
I seems to remember a distinct time during which Claude for example became incomprehensible and weird with its writing, whereas before it was reasonably OK, even if still not that good
Especially for general purpose technical writing, system design, architecture and code comments, I do use pstack unslop [0] and technical-writing [1] skills. They do wonders.
[0] https://github.com/cursor/plugins/blob/main/pstack/skills/un... [1] https://github.com/cursor/plugins/blob/main/pstack/skills/te...
Ease of comprehension is not prioritised in training. I doubt frontier labs would spend enough money to train on something that is non-measurable and subjective.
There's so much fluff because it doesn't care about what the user actually cares about.
So there's the usual structure of "it's not x it's actually y" signs.
One factor might be this: LLM's know orders of magnitude more stuff than humans. They are experts in whatever subject is being discussed at the moment. I think they often respond as if the human user has a similar baseline of factual knowledge in every field.
People who are knowledgeable in a particular obscure area tend to do the same thing.
It's painful enough that I find myself now consistently asking for "8th grade reading level", "children's story book", or now "ASD-STE100" English in all my prompts. Somehow just that tiny bit of steering constrains the language choices significantly.
This is an absolutely critical issue; it's impossible to work with AI that can't communicate coherently. It's barely usable for reasoning on complex issues because of the language problems described exactly in that synopsis.
Because AI's language isn't a language at all, but its imitation. Imagine you're given a map skillfully created by someone who has seen many maps, but never bothered to understand their meaning. How much time will you waste before realising that this map is an imitation?
It's simply because AI has no motivation, no care, no desire to say anything, so how can it burn with a passion to make its point?
We had a funny chat at work. Two of us were light-heartedly joking about how you never paste the AI at people.
A third team mate uses a voice dictation tool to get down the basic ideas and then massages it with AI, but these two guys rib each other about it.
Another my other team mates (team mate 4 in this story) sheepishly turns around and asks if people can tell when he does it? (It's EXTREMELY OBVIOUS that he isn't writing anything) I confirmed it, in a friendly way. He defensively said he tries to get the AI to use his own voice.
I casually mentioned that "writing is thinking" or something like that and how I would prefer to think the thoughts. He didn't say much.
I hate AI writing. So much.
The text data used in the training of large language models often reflects human expression, which tends to elaborate on context and express precisely. This results in the model's generation tending towards complexity, rather than serving human communication in a conversational manner.
I'm just glad it's not just me being stupid. Sometimes I have to read a sentence several times and then ask to elaborate because I can't comprehend.
Maybe I'm the odd one out, but I really don't find AI generated language, in almost any model, to be hard to understand. I'm not trying to be snarky or annoying or pompous or whatever word might be "this guy over here think's he special."
It uses a lot of metaphors. It references stuff implicitly - a LOT - and assumes pretty much all the time that the thing it did is what you saw, which you didn't. But if you just.. read the stuff it says, and know a few old-school sayings and thesaurusize your reading style, it usually says things quite meaningfully.
I think a lot of people just don't read what it writes. So many folks I've watched used LLMs don't write in complete sentences, don't read the sentences that come back, don't watch the turns and tool calls in the middle, and then just respond to the message as if it says something it doesn't. I'm saying all this before I read the paper, but I've done a lot of personal and team experiments with this, so I'm quite curious to see what they say. I'll edit this after I go through it.
--
Fascinating! I did actually run into a lot of this with my personal 'engineering and communication styles' experiments. Most of the papers out there talk about models moving toward human language inside of a context window, and not the other way around (much smaller effect for human->model). I had a few years of conversations I ran on local and 'frontier' Claude and found that, yes, in general, information density is what each response turned out to be geared toward, whether it was everyday health discussions or deeper engineering topics and research. The component tests they ran I've been exposed to already, but I think their conclusion nailed it: ```This discrepancy may help explain why many frontier LLMs use linguistic constructions which are technically allowed by the grammar of English, yet are out-of- distribution for human language and are associated with comprehension difficulty.```
Indeed. I just struggle to revise it when generating website/social media post copies, more so now than before from what I can recall.
My experience is: The LLM fails to "understand" or "intuit" who the audience is. It tends to reward its own recent context and its interaction with the prompt writer, rather than "considering" documents from the viewpoint of a hypothetical reader. You can often prompt them to go more in that direction but it gets tiresome.
The best approach is just to write the outline and initial text in a very messy fashion yourself. And then have it clean it up and add detail. And then edit again by hand.
This was excellent, and one would hope they can at least try to optimize these metrics a bit.
I have seen people saying this for a long while now. I don't know how to say this but I understand Ai stuff better than non-Ai. When I see people complain ; I always wonder why because I think that its pretty good with my own senses.. Much much better than my writing skills... My brain has been tuned by computers I suspect .
I have zero issues understanding what they write.
I know the paper specifically calls out Opus 5.5 and Opus 5, but I can’t see different versions of Sol, or Astra.
I think the difficulty in comprehension was mostly an Anthropic problem… partly because Anthropic employees stopped reading Opus output around the time Fable came out and stopped noticing RLVR going off the rails with compression.
The paper was a good read, in any case. Maybe I gotta start sending this article back to LLMs when they start compressing text too much :)
I find AI generated language hard to understand when it is "in it's own world" and it knows what it means but can't articulate it without buzzwords and slop. A follow up "ELI5" type of prompt often sorts this out. Seems like a failure mode and you need to snap it out sometimes. Opus 5.5.
Article seems to answer the question: because generated text has the competing objective of information density and supporting reasoning chains.
But I also wonder why AI loves to add slogans, catch phrases, and witty sentence fragments. It’s grating and a dead giveaway of slop.
Thank gods I'm not the only one noticing this! When humans write impenetrable prose, most of the time it will be too verbose. Big words, too many words, long sentences, being parenthetical, that kind of thing. LLMs, however, make their writing hard to understand by being too terse in a peculiar way, where "the caveman" is also "the mad scientist". I've noticed both Claude and Codex often invent their own jargon to explain what the code does and what their fixes are about. It's irritating, but I thought maybe I was getting senile and could not understand a simple sentence.
A: Exhaustion.
this is hard to undersand
AI language will converge towards maximum entropy, like compression, due to cost pressure to conserve tokens. LLMs are not limited by the human 5+-2 working memory
hmmm.that’s exactly how I felt. The language itself has to be written briefly, and multiple pieces of information are compressed into the text
[flagged]
[flagged]
As a technical writer, I believe default AI responses are optimized for what people _think_ is good writing rather than what actually is good writing.
My guess is years of RLHF optimized for rater's first impressions of how well-written something was. It uses lists! It has headings! It uses fun phrases like 'smoking gun'! It's long and informative! These things are relatively fast and easy to identify and select as a preference.
But none of this really covers whether the response is _useful for the audience_. If you're not the audience, it can be hard to tell what is and isn't useful.