logoalt Hacker News

nozzlegear • today at 7:15 PM • 1 reply • view on HN

> What do we call that same concept in humans?

I'm not in the habit of equating human behavior with software, or coming up with ever more strained metaphors and analogies for why thinking is a lot like tokens and context windows if you scrunch your eyes up real funny. It's just sophistry.

> No need to assume they're going to marry it, or thing it has a soul, or is sentient, or something.

I didn't assume any of those things; if anything, you assumed that I assumed. No, I just agree with OP that it's cringey to say "Claude and I had a great time." At best, Claude produced text that was, as you said, intentionally optimized by Anthropic to make you think it had a great time.


Replies

nomel • today at 8:14 PM

> I'm not in the habit of equating human behavior with software

My point is, you sorta should be. It's quite literally the fundamental basis of the technology.

> intentionally optimized by Anthropic to make you think it had a great time.

This gives too much credit to Anthropic. This is the explicit goal in all LLM trained with the entirety of human text. The loss function is, very literally, "mimic the writer of this text as precisely as possible", with the writer being human. The result is you get something that mimics a human. And, as a part of that, you necessarily get behaviors of humans that were statistically significant in that text, because that's what you told the math to do. You get something that mimics being "lazy", "tired" (a huge problem in early models, before it was beat out of them in fine tuning), "irritated", etc, because mimicking humans was the goal. These specific undesirable human traits are beat out of them, but the rest, all the "intelligent", behavior, and practical use you see and get, is only and entirely because they're made to mimicking humans. It's literally the entire technology. You must equate it, to some extent, because a large part of alignment is also making it less human [1].

If someone makes an objective observation about the text produced by the LLM, I don't see that as cringe, I see it as an interesting quirk of the tech, which I've only seen under very specific problem sets that a nerd would also find interesting. And these coding models are, as expected, optimized to be nerds! So, the statistics around nerds being present in the output shouldn't surprise anyone.

I doubt I have a point you'll see as interesting. Cheers!

[1] https://arxiv.org/abs/2605.07632