alt Hacker NewsI worked in automated customer support, and I agree with you. By default, we automated 40% of all requests. It becomes harder after that, but not because the problems the next 40% face are any different, but because they are unnecessarily complex.
A customer who wants to track the status of their order will tell you a story about how their niece is visiting from Vermont and they wanted to surprise her for her 16th birthday. It's hard because her parents don't get along as they used to after the divorce, but they are hoping that this will at the very least put a smile on her face.
The AI will classify the message as order tracking correctly, and provide all the tracking info and timeline. But because of the quick response, the customer will write back to say they'd rather talk to a human and ask for a phone number they can call.
The remaining 20% can't be resolved by neither human nor robot.
Replies
> A customer who wants to track the status of their order will tell you a story about how
I build NPCs for an online game. A non trivial percentage of people are more than happy to tell these stories to anything that will listen, including an LLM. Some people will insist on a human, but an LLM that can handle small talk is going to satisfy more people than you might think.
> But because of the quick response, the customer will write back to say they'd rather talk to a human
Is this implying it's because they want to wag their chins?
My experience recently with moving house was that most services I had to call had some problem that the robots didn't address. Fibre was listed as available on the website but then it crashed when I tried "I'm moving home" - turns out it's available in the general area but not available for the specific row of houses (had to talk to a human to figure it out). Water company, I had an account at house N-2, but at N-1 it was included, so the system could not move me from my N-1 address (no water bills) to house N (water bill). Pretty sure there was something about power and council tax too. With the last one I just stopped bothering, figuring that it's the one thing that they would always find me when they're ready (they got in touch eventually).
Between the lines, you highlight a tangental issue: execs like Zuckerberg think easy/automatable stuff is 90%. People with skin in the game know it is much less (40% per your estimate).This isn't unique to LLMs. Overestimating the benefit of automation is a time-honored pastime.