Human insight can emerge from attacking the semblance of sense in the LLM's output.
Much of the cognitive work in writing emerges in the labor of writing prose and challenging assumptions against the model you build in your head as you go. I've found this process is inverted when working with an LLM: it in effect emits a provisional structure first, and I discover what I actually think by finding where its output is vague, overconfident, incomplete or outright false.
Accepting surface-level coherence as finished thought is the failure mode to avoid.
Sure this is true. And if an AI draft helps in creating quality writing, great!
I was thinking more when it comes to stuff other people send me (like a freelance writer) and “editing” is basically writing the first draft because no cognitive effort has yet been spent on the piece
Yeah I've written some docs using the following process:
1. Throw some notes, PR links, etc. into copilot and ask for a first draft
2. Go back and forth for a while asking for revisions
3. Trash all of the AI-generated text and rewrite the whole thing myself from scratch, which I can do quickly in a typing frenzy because I now have a clear idea what I want, by way of seeing what I don't want.