These are all important points and I love the analogy. But there is an even bigger issue with having LLMs write for you:
Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.
Reviewing gives you the chance to ensure the arguments connect solidly, that references are accurate (even informal references) and gives you the time to consider counter-arguments you aren't addressing.
None of this matters much on LinkedIn, but it matters a lot in our work. You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.
(I work at Anthropic) I agree. I use an LLM to write my code, but I do all of my writing by hand, since it helps me think.
It reminds me of the transition over the last year from AI-assisted coding to AI doing all the coding. At first the code output wasn't good enough, and humans read and iterated on the code all day, so the details of the source code mattered. Now, the code is largely high quality and it meets a large set of guardrails we've set up over the years (linters, typecheckers, security checks, LLM-assisted code quality checkers), and it's just Claude working on the code, so the details matter less and engineers think a level or two up (machine code < assembly/bytecode < source code < conversation with agent < artifact with high level design).
I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document. But writing and coding are different enough in a number of ways that this is far from inevitable.
I've thought exactly that "writing is thinking" before as a reason to not let a LLM write for me.
Then again, I've seen a counterargument [1] by someone who clearly heavily uses LLMs for writing (going by both their LLMy writing style and their own admission). The person I'm citing describes a process where they get a LLM to write something, they check over it and provide feedback to the LLM, the LLM rewrites, and the process repeats iteratively. So clearly he is putting thought into the process.
I think there is something valuable missing, even if it's hard to clearly express. I'll try. The threshold for what I'm willing to accept if I'm simply approving something is likely different from what I'll get if I write something myself, for instance. Saying "LGTM" is too tempting. It seems to me like he's outsourcing his selection of topics to cover as well. If you're not thinking yourself about what to cover then it would be very easy to miss a critical subject. There also an asymmetry between checking and generating something with constraints placed on it. Checks can't catch everything, and a constrained generating process can reduce the amount that needs to be checked, avoid issues that can't be checked so easily, and focus your attention on areas that you know historically have had issues with this generating process. I've thought about this quite a bit in terms of whether to write new code or use an existing library. Sometimes "the devil you know" (my code) is better than an existing library simply because I understand its flaws better.
Same. I do not use LLMs for writing and I recommend everyone i work with to do the same. Even for code, I come to the code with the idea of what I want and provide that to the agent, then we iterate on what needs iteration or decisions.
Just did this for some caching, started with the structure of what I wanted the cache to look like, asked the agent to start the work, didn't like how the architecture came about, scratched it and rewrote the whole thing, so it fit the model I now wanted.
Was also just having this discussion with friends, that I can only think seriously about a subject if i can put it to paper (even virtual paper). Writing lets me organize my thoughts, clearly define my assumptions and see if any of it make any sense. I can't imagine what it would be like if i couldn't write, my brain just doesn't work without it.
I have found this extremely relevant as a (primarily) non-verbal thinker.
I don't, generally, think in words, more in - I guess I would call it something like meta-shapes? A sense of a shape but not things I can exactly visualise.
(You might be surprised to read this and then hear I have an English degree. Surely I thought about Shakespeare in words?! Nope. Shapes, movement, structures)
For me, having to write is critical because it is the only way I practice serialising my thoughts in a way other people can understand.
If I do not then I get very "deep" into my own way of sensing ideas and it's difficult to dig myself back out.
This might also be why I have never been very enchanted by LLMs? They only seem to "think" verbally. So it is always a translation effort for me.
I never can really enter any "flow" state with an LLM. My intuition is that highly verbal thinkers can enter flow with LLMs very easily
This is why I like fountain pens… I regularly write down the things that are important to me.
Things I need to do, ideas I want to ponder on, people I need to remember or get respond to.
Writing is learning. Writing is understanding.
I have so many conversations with people who are always telling me I’m “retro” or “old school” for doing this.
I don’t even bother explaining the psychology behind it anymore. I’ve got no time for the ignorance.
Yeah. I've been using AI a lot in my projects but I still write code and especially the documentation and articles for my website. It's slowing me down a lot compared to just letting the AI rip through projects but I think it's worth it. I see it as distillation of the AI's weights into my own brain.
Related, one of my favorite quotes by the computer scientist Leslie Lamport is this one:
"If you’re thinking without writing, you only think you’re thinking."
> Perhaps I'm just too painstaking a type of person, but I can't grasp much of anything without putting down my thoughts in writing, so I had to actually get my hands working and write these words.
> As I write, I think about things. As I write, I arrange my thoughts. And rewriting and revising takes my thinking down even deeper paths.
- Murakami, "What I Talk About When I Talk About Running"
I always found writing to be an unnecessarily arduous tool for communication, so I never do it until I've overthought what I want to express and approximately how. At that point I flesh out the skeleton of the message, and pile on words and structure and references and other rhetorical tricks until the packaging feels sufficient. Rarely, if ever, have I experienced writing things down altering my perception of the strength or weaknesses of the chosen arguments, or revealing new ones. That is to say, I really don't vibe with the concept of writing being thinking, rather it's a waste of humanity's resources to push it as The Tool.
> Writing is thinking.
This way of phrasing it is needlessly confusing. Writing is a kind of thinking--one of many--but it's not equivalent to thinking.
Dialectical thinking, for example, produces similar results.
On the other hand there is so much things that don’t require hard thinking. An example is I got an email the other day from a supplier asking if they can turn off their old email. It didn’t require ”thinking”. All I had to communicate was ”turn off graphql but don’t touch restapi”. Instead of sending off such a short and maybe unclear reply I had AI type up a concise and clear reply with exactly what can be turned off and what must be left on. Could I have done it on myself? Sure but would require more work than just a quick prompt and copy paste.
Sometimes. Other times writing is just communicating thought that already happened. Say for instance a weekly status update. Maybe you get something out of that, but frequently you don't get much or anything.
Yeah, I often find that I think I understand something perfectly until I try to explain it.
That's an important point. It's the core of your life, thinking, balancing, exploring, improving. I don't believe adjusting something large you didn't do will ever benefit people.
This is perhaps even truer for writing code.
I often write a reply to a HN post, just to think about the topic or idea and decide how I feel about an issue. More often than not, I just delete the post without ever having submitted it.
writing also implicitly has your review baked in. hand written code is "reviewed by construction", if you will.
I completely agree. And I'd say that at least in my case, it applies to both writing prose and code
I don’t think it’s a good analogy.
Having your fly open is a harmless mistake that has little impact on your peers. People may or may not mention it to you but it’s not something they’ll hold against you.
Posting LLM slop under your name is a deliberate act. You decide to damage your message by taking a shortcut.
The obvious analogy was speaking while chewing.
This puts into words something I have been thinking about but haven't been able to articulate when people ask me if I'm using AI heavily. Usually when I write a report, I start out with a question and try to find some hints that will give me hypotheses that I can then test. I can't just prompt "Write me a report". The process of coming up with the information in the prompt is best done by writing the friendly report.
They are powerful tools but they do not have any human understanding - that isn't their optimization target.
Ofc the rest is all right on, but I'd quibble with this specific idea. LLMs are absolutely targeted at modeling human understanding, which is the same faculty that contains what we call perception (!= sensibility) and intuition (!= rationality). It would be nice to train them to be completely alien from the ground up, butA) we only know of one species capable of metacognitive understanding,
B) we already tried that in the 1970s, and it was good work but often evolved into what we'd call boring ol' computing rather than AI, and
C) an alien mind wouldn't be a very good agent, for a ton of reasons relating to affect, conversational rythyms, cultural understanding, etc.
The trick is to make something that acts like a human but with the affordances of a computer (e.g. scalibility, symbolic certainty), without making it so human that it takes issue with its existential reality and/or use of its labor...
Writing can be thinking. There's a huge presumption that if someone is banging away on a keyboard they're doing work because you can hear and see them doing stuff. But that's the whole plot of The Shining -- Jack despite all his writing wasn't thinking at all.
Likewise using AI can be thoughtless, but it doesn't have to be. I don't see why a valid creation process can't be like this Simpson's meme[1], where you start with a rough object and then cut away and refine until it's done. I don't see it as lacking merit or requiring less thinking compared to starting from a blank canvas and adding more until it's done.
And either way at the end of the day the writing artifact stands on its own. It's either good or bad, taste permitting, and can be evaluated for what it is.
[1] https://media.licdn.com/dms/image/v2/D4D22AQFoqRgMxteTNg/fee...
> You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.
Understanding is the bottleneck; the way they speed things up is by letting me outsource understanding, and get back a summary. The entire advantage to AI is that it lets me skip understanding the problem, and just get a working solution.
> Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.
I don't disagree, but I think it's often not appreciated how much there's other work to writing too.
The biggest one is that you have to communicate non-interactively to an unknown audience. Having to (literally) put it in someone else's assumed terms does help giving different perspectives into the matter, but doesn't necessarily help one's own thinking that much. Instead you have to do some of the reader's thinking for them.
You also have to spend time on textual matters like grammar and style and a lot of "unspoken rules", which aren't really about linearizing your thinking about the contents.
Not all writing is thinking and not all thinking is writing.