So what does this mean in practice, though?
Let's say you are correct.
You ask an LLM to write something for you, and to you it looks really, really good. So based on your conjecture, that means I am not a very good writer.
Ok, but how does that change what I should do? If I am not a very good writer, that means an LLM IS actually better than me, even if it might not be objectively good to an expert writer.
My two choices are to keep producing my own crappy writing, or use an LLM to create better (but not great) writing.
Wouldn't it make sense to use an LLM?
It seems to me your premise leads you to the same conclusion you would reach even if your premise was false; if me thinking an LLM is good at a task means I am very bad at that task, I am probably better off having an LLM do it. On the other hand, if you are wrong, and I think an LLM is good at something because it actually IS good at that thing, then I should also use the LLM to do the task.
Either way, the LLM is better than me at the thing.
Well, practice to get better at writing (and, therefore, judging writing) yourself. It seems obvious. Your skills are not frozen in time and set in stone.
Well, the third choice is to develop as a person and become better at writing. Which you do by doing some crappy writing and learning from it.
>If I am not a very good writer, that means an LLM IS actually better than me
If you're not a very good writer, I'll at least skim your work to see if it contains any good ideas. If it's slop, I'll just close the tab. You already told me it's not worth caring about, so I'll agree with your decision.
> My two choices are to keep producing my own crappy writing, or use an LLM to create better (but not great) writing
Are you also an LLM or do you have the capacity to learn and grow?
I mean, you have the ability to learn to do stuff better to a certain extent, so it's not like your only choices are "suffer through the writing I'm capable of producing today for the rest of my life" or "give up on ever writing anything myself". Writing stuff yourself is pretty much a requirement of getting better at it, and arguably even if you do intend to use LLMs to supplement it, having a better baseline will be valuable for additional iteration with the LLM.
Well, no. The LLM is just better than you in a narrow band that appears wider to someone below than above.
From above, or from below with adequate exposure, it feels facile and hollow. It is good at weaving grammatical structures. It is not good at thinking in words in a way that invites a fellow human along for the journey. Because it doesn't think.