alt Hacker NewsUnderstanding what is going on with AI productivity is … frustrating to say the least.
The best I can say is that genAI is a self reported a 20% efficiency boost, and for a very (very) small group of people, it’s maybe a 2-3x boost. (And if you are at a frontier lab, you go fly into the big bucket of exceptions)
At this point, for most use cases, AI productivity is either the equivalent of giving people 3D printers, and seeing little benefit, or signing up for an outsourcing service, just without the development of human capital anywhere.
Replies
I think it depends on how you measure the boost. If you are talking about generating a first draft then yes, the boost is there. If you’re talking about completing the project in all well tested and architected aspects, then overall there really isn’t a boost.
6 hours of debugging and docs reading is not equal to 6 hours of prompt fiddling. The return of value beyond the few fixes applied will be almost nil from the fiddling.
>Understanding what is going on with AI productivity is … frustrating to say the least.
Agreed. I think one of the hardest things about it is that productivity != value. You can push all the code you want, but if it's not driving revenue up or cost down, it doesn't matter economically.
Here is the best data I've been able to find. An observational study of 4000 teams over 2 years across many different organizations. Data gathered from their task management, version control, and CI/CD tooling. Critically - this is not survey data. It's much more direct measurement.
https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways
Faros argues that teams are seeing about a 16% throughput improvement (PR merge rate) with heavy AI use.
I argue here that their data actually indicates negative absolute impact on throughput.
https://unessays.substack.com/p/talk-is-cheap