>I keep hearing this claim, and yet I keep seeing LLM output which is trivially distinguished from human writing.

That's mostly true for longer LLM output with all the sycophancy / LinkedIn bias thrown in.

Make it casual conversation or comments, and give it instructions on appearing casual, or even better kill the censoring and fixed-prompt (with an open model), and it's orders of magnitude more difficult, unless if you suspect it and try specifically tailored prompts to sniff it.

There's no shortage of people obliviously discussing with AI bots in comment sections.