This bolsters OP's point.

It's the same as calling a gun a "powerful hole puncher".

There is a reasonable objection that a gun is such a powerful hole puncher that it is not merely a hole puncher. But the clear implication of that objection is that the user of the tool now has more responsibility and that the tool should be treated with more respect/care.

LLMs are a tool. The impact of using that tool is the responsibility of the end-user. As the tool at hand becomes more powerful, the care with which the end-user should treat that tool increases.

For some reason, with LLM-based systems, we seem to be going the opposite direction. As the tool becomes more capable people absolve themselves and others of more responsibility. This feels backwards to me.

(Aside: in a lot of ways, at least form a scientific and engineering perspective, modeling LLMs as "fundamentally auto-complete" is an incomplete theoretical model but one from which we can still get a lot of mileage.)

Between driving a car and driving a forklift, which of them would you like to see regulated more heavily?

You don't get it. A human set up a software system allowing spicy autocomplete to solve open math problems if the appropriate keyword appears in its output.

“Autocomplete” does not represent an analysis of its problem-solving capability, but of its place in the social order and its expected social competence.

I don't spend much time interacting with zoomers, but I'm still surprised that "spicy $foo" sends fellow boomers through such a loop. I didn't have to puzzle it out, it was fun juxtaposition wordplay and when it's deployed well I still find it amusing.

> the spicy autocomplete can solve difficult open math problems

No it can't. It can't even solve my son's 4th grade math homework. (This is a real use case for me, not a dumb benchmark.)

You just know nothing about math and are happy to parrot bullshit AI salesmen are selling you.