alt Hacker News> [...] he had intervened at forces that were deploying commercially available AI tools before they had been properly assessed [...] “All forces have got a good policy on the use of Copilot,” Murray said. “All forces will have a policy that says, ‘Check everything that it produces’.”
Not only are they using AI before they've properly assessed them, they also end up using Copilot which must be one of the worse AIs currently available, probably because of existing Microsoft relations. And on top of all that, they hope to be able to rely on "Please review the outputs" which obviously isn't an actual solution here, of course people will get complacent and throw stuff over the wall whenever they can.
Replies
Something happening in the US right now is that the "presumption of regularity" is being openly challenged by judges. To the best of my understanding , it's the presumption that the testimony of the police is truthful until proven otherwise.
I think "check everything that it produces" will ultimately have to happen in cross examination on the witness stand. "Did you use AI" will be the first question.
The mindset must be that if you use AI (which I happen to advocate for) you are also responsible for the output, if you use the output publicly. AI is obviously very powerful if used responsibly - the human is responsible for it once it is used - however it’s used.
> on top of all that, they hope to be able to rely on "Please review the outputs" which obviously isn't an actual solution here, of course people will get complacent and throw stuff over the wall whenever they can.
This is honestly the fundamental problem of AI as I see it
When we offload our work to a different person we can calibrate our expectations to our past experiences with that person. With AI the experience is not very consistent. To use AI effectively you basically should treat it as a low trust, brand new coworker every single time you use it
That doesn't really scale, so people have two choices: be constantly hyper vigilant for mistakes the AI makes, or become complacent and trust it more than they should
People rightly point out that humans make mistakes too, not just AI. But humans have a pretty manageable cap on the amount of output they can produce. One human can pretty thoroughly review the outputs of a small team of other humans
One human can't possibly thoroughly review the volume of output that an LLM they are prompting can produce
> “All forces will have a policy that says, ‘Check everything that it produces’.”
Everyone I talk to (including outside of tech) is going through this phase at their companies. It’s not working.
Checking the output seems like a simple request, but the question becomes: Check against what? If the police are making a document that sources from another report that another officer used AI to produce from their notes which were also run through AI and on and on, an inconsistency that leaks in at a previous step will check out when someone reviews the output against the inputs.
We’re all also discovering that many people’s idea of reviewing the output is to skim it and verify that it looks convincing enough. Checking facts is hard and takes time. These people are using AI because they want to work less, not to give themselves extra work.