alt Hacker NewsSelf-contradictory policy.
> Reporters may use AI tools vetted and approved for our workflow to assist with research, including navigating large volumes of material, summarizing background documents, and searching datasets.
If this is their official policy, Ars Technica bears as much responsibility as the author they fired for the fabricated reporting. LLMs are terrible at accurately summarizing anything. They very randomly latch on to certain keywords and construct a narrative from them, with the result being something that is plausibly correct but in which the details are incorrect, usually subtly so, or important information is omitted because it wasn't part of the random selection of attention.
You cannot permit your employees to use LLMs in this manner and then tell them it's entirely their fault when it makes mistakes, because you gave them permission to use something that will make mistakes 100% without fail. My takeaway from this is to never trust anything that Ars reports because their policy is to rely on plausible generated fictional research and their solution to getting caught is to fire employees rather than taking accountability for doing actual research.
---
Edit: Two replies have found complaint with the fact that I didn't quote the following sentence, so here you go:
> Even then, AI output is never treated as an authoritative source. Everything must be verified.
If I wasn't clear, I consider this to be part of what makes the policy self-contradictory. In my eyes, this is equivalent to providing all of your employees with a flamethrower, and then saying they bear all responsibility for the fires they start. "Hey, don't blame us for giving them flamethrowers, it's company policy not to burn everything to the ground!". Rather than firing the flamethrower-wielding employees when the inevitable burning happens, maybe don't give them flamethrowers.
Replies
Is this a case of Moral crumple zones? where "responsibility for an action may be misattributed to a human actor who had limited control over the behavior" https://www.researchgate.net/publication/351054898_Moral_Cru...
The next sentence after your quoted section:
“Even then, AI output is never treated as an authoritative source. Everything must be verified.”
> permit
I suspect that it's more like "ordered."
Many companies are now requiring staff to use AI.
Of course, if the employee screws up, then The Secretary Will Disavow Any Knowledge of Your Activities.
> ...this is equivalent to providing all of your employees with a flamethrower, and then saying they bear all responsibility for the fires they start.
This is essentially the policy of most SWE groups but with PRs merged, though right?
You can / should use AI to accelerate SWE workflows and assist in reviews but if you merge something that is bad or breaks production that is on you.
> "Hey, don't blame us for giving them flamethrowers, it's company policy not to burn everything to the ground!".
Flamethrowers are inherently dangerous to the operator and are ~intended to be used burn things to the ground.
I'm no expert on arms but there is probably a simile with a better fit out there.
I'm not a journalist and just for random things I'm interested in, I have no problem using an LLM to point me in a direction and then directly engage with the source rather than treat any of the LLM output as authoritative. It's easy to do. This is not a flamethrower.
I have a little bit of a bias here, as I am building forth.news, which is an AI-powered news platform -- but I am also a former journalist.
It's not necessarily contradictory. I see this more like giving your employees cars, but telling them they are responsible if they get into accidents.
All of this is entirely predicated on expectation and responsibility. First, mark something as being AI if it cannot be verified, and verify everything that you can.
Forth is using AI so we can detect and push out stories as quickly as possible, getting breaking news out there as soon as it breaks. Our summaries are AI, but marked as AI. Our underlying source information is right there and cited. We try to be as transparent as possible about the tools we are using, and the tradeoffs.
Every journalist should instinctively and reflexively double check everything, regardless of the source. There's an old maxim, "if your mom tells you she loves you, check it out." Being from an LLM doesn't change that.
> the author they fired for the fabricated reporting
Didn't one of the magazine's editors share the byline?
I've never seen a summarizing mistake from any modern LLM. What are you even talking about?
LLMs hallucinate when they don't have enough context. Not when they're just cutting down the message in their context.
They're also allowed to use wikipedia to use research. It has similar sorts of problems.
> If I wasn't clear, I consider this to be part of what makes the policy self-contradictory.
Yeah, it's not. And this is a bad take. Adults can easily do this. That you consider this to be burning everything to the ground says more about you than anything.
> LLMs are terrible at accurately summarizing anything.
I think you are perhaps stuck in 2023?
[dead]
[dead]
[dead]
> LLMs are terrible at accurately summarizing anything. They very randomly latch on to certain keywords and construct a narrative from them, with the result being something that is plausibly correct but in which the details are incorrect, usually subtly so, or important information is omitted because it wasn't part of the random selection of attention.
I don't know what you've been doing, but the summaries I get from my LLMs have been rather accurate.
And in any event, summaries are just that - summaries.
They don't need to be 100% accurate. Demanding that is unreasonable.
> You cannot permit your employees to use LLMs in this manner and then tell them it's entirely their fault when it makes mistakes, because you gave them permission to use something that will make mistakes 100% without fail.
Yes you can. The same way Wikipedia (or, way back when, a paper encyclopedia) can be used for research but you have to verify everything with other sources because it is known there are errors and deficiencies in such sources. Or using outsourced dev resource (meat-based outsourced devs can be as faulty as an LLM, some would argue sometimes more so) without reviewing their code before implemeting it in production.
Should they also ban them from talking to people as sources of information, because people can be misinformed or actively lie, rather than instead insisting that information found from such sources be sense-checked before use in an article?
Personally I barely touch LLMs at all (at some point this is going to wind up DayJob where they think the tech will make me more efficient…) but if someone is properly using them as a different form of search engine, or to pick out related keywords/phrases that are associated with what they are looking for but they might not have thought of themselves, that would be valid IMO. Using them in these ways is very different from doing a direct copy+paste of the LLM output and calling it a day. There is a difference between using a tool to help with your task and using a tool to be lazy.
> it's company policy not to burn everything to the ground!
The flamethrower example is silly hyperbole IMO, and a bad example anyway because everywhere where potentially dangerous equipment is actually made available for someone's job you will find policies exactly like this. Military use: “we gave them flamethrowers for X and specifically trained them not to deploy them near civilians, the relevant people have been court-martialled and duly punished for the burnign down of that school”. Civilian use: “the use of flamethrowers to initiate controlled land-clearance burns must be properly signed-off before work commences, and the work should only be signed of to be performed by those who have been through the full operation and safety training programs or without an environmental risk assessment”.