alt Hacker News> when you only read the threat by spying on them
The AI companies have clauses in their user agreements saying they can review content flagged as harmful. It’s not legally spying.
If you recall previous outrage about ChatGPT being used in cases of suicides or shootings, this is the result. Every time a crime was committed and the police found ChatGPT history about the crime, the media turned it into a frenzy. So the AI labs added safety filters to their consumer plans that detect threats of violence, escalate them to human review, and report to the police.
Spying is not the right analogy because the information was given to the police by a third party which had a EULA saying they would do this. A more analogous situation would be someone reading another person’s diary and then turning it into the police department. There might be some limitation in the law that makes the evidence inadmissible because it was not intended to be shared with anyone, but that’s a separate decision.
Replies
It kinda is, actually. If the LLM had responded with 'woah, are you serious? That sounds like a crime and I can't just ignore that, it's made clear to the customer that such statements are out of bounds even if they were meant hyperbolically or humorously. But if someone crosses the guardrails and the system silently reports them, that's very much spying.
Obviously, it's hard to judge exactly what was appropriate there because we're being asked to extrapolate from a two word quote about the customer intending to "shoot up" the sheriff's office. Consider the following two statements, which express quite different levels of intentionality.
I got a $200 ticket from a sheriff's deputy today for throwing away an apple core. I'm so mad. I'd like to shoot up their office!
Those sheriff's deputies have exhausted my last reservoir of patience. I'm going to shoot up the department. They'll be sorry when they're sprawled all over the floor bleeding out from saucer-sized shotgun slug wounds. I can't wait to hear the screaming and crying of their miserable families!!"
I'm guessing that the diary entry was a more casual expression similar to the first statement, or they police would have quoted more of the statement to emphasize the apparent severity of the risk but it's hard to say without reading the charging documents.
>Spying is not the right analogy because the information was given to the police by a third party which had a EULA saying they would do this. A more analogous situation would be someone reading another person’s diary and then turning it into the police department.
This is spying with extra steps couched in corporate speak.
Many clankers deny data retention or spying on the user if you ask them. That should be completely illegal.
Then, you can write anything in an EULA but it is not automatically legal either.
Not only that they can review flagged content, but they tend to have separate retention policies for flagged content. Anthropic's is this: "We retain inputs and outputs for up to 2 years and trust and safety classification scores for up to 7 years if your chat or session is flagged by our automated trust and safety systems as violating our Usage Policy."
So don't run for office or anything like that. Someone, somewhere will have a contact that will get that.