No AI company is addressing the elephant in the room that you need someone experienced constantly monitoring any agentic workflows. This means that the cost savings of agents are a myth.
My company actually did an internal study of agent usage for coding and found it only improved productivity by 10-20%, basically on the same level as good code templates or an autocomplete.
> My company actually did an internal study of agent usage for coding and found it only improved productivity by 10-20%, basically on the same level as good code templates or an autocomplete.
That's still a pretty good outcome. 20% more output across a company is huge when you think about it. Definitely not going to change the world completely though.
> No AI company is addressing the elephant in the room that you need someone experienced constantly monitoring any agentic workflows. This means that the cost savings of agents are a myth.
I mean, it depends on the agentic workflow. Like for production code, definitely. For document and claim review, you probably need a targeted sample on a daily basis but you get massive gains.
Less and less true with every new generation of AI systems.
AI gets better and better at operating self-supervised, and the amount of skill needed to supervise an AI in a useful fashion only ever goes up.
I maintain a part of my team's CD process and I've observed a 30% increase in PR velocity since we started adopting agentic tools but it was a "one-off" increase (as-in, it hasn't continued to increase beyond that since about a half-year ago).
I'm guessing though that there are other improvements in code quality and feature velocity. I've noticed personally that AI is really good at catching smaller things that are easy to miss (e.g., if you ask it to rename fooTheBars it also updates all the relevant comments or enums that you might have missed).