alt Hacker News+1 it’s a tool
It’s not perfect, it has shortcomings, it sometimes produces bogus outputs. All of that is fine for a tool, it’s not fine when it pretends it’s a conscious being, because errors start to feel like lies and it becomes a bit too personal.
Replies
barnabee • today at 12:18 AM
I think the problem is that we’ve anthropomorphised the LLMs in their training and through system prompts to the point that that seems reasonable.
We don’t thank our other tools like grep, or the compiler.
If you train your LLM to produce words that seem like human responses or conversations, they will. If you train them to seem like god-like superintelligences to people who think that’s what they’re creating, they will.
People want it to be Data from Star Trek, when it really should be the ship's computer. I want to tell it to run a simulation accurately, create some solved tool, etc.
I don't think there's any correction that can return LLMs to a purely tool-space. Too many AI boyfriend/girlfriends.