logoalt Hacker News

Applejinxtoday at 1:58 PM1 replyview on HN

I asked some AI-using compatriots a while back who were complaining about this, 'isn't it doubling down on bullshitting you?' and got some pushback along the lines of 'it isn't a person therefore doesn't have dark motives like that therefore can't be doing that to us'.

Didn't convince me. I think bullshitting like this can be a behavior, not just the intention of a human. If it's blowing a lot of smoke to use fancy words and phrasings (and semicolons! All the trimmings) it's fair to ask if it's systemically bullshitting you: i.e. the behavior is meant to have you shut up and trust it and not ask questions.

Who's driving that is still important: if the company's directing it to do that in system prompts that are adversarial to users, that's a big yikes. If it's an epiphenomenon of the company demanding it get ever smarter, maybe it's a sign that their demands are not having that result, rather they're making it bullshit more explicitly and mimic more 'smart' signifiers.


Replies

whstltoday at 3:54 PM

> the behavior is meant to have you shut up and trust it and not ask questions

This seems to be exactly the kind of thing automated/massive training would produce, just like it did with sycophancy recently.

Claude users would just gave up after the word vomit and some classifier considered it a success and into the model it went.

Wrong incentive and nobody checking.