A few months ago I had a conversation with a friend who runs a small company doing about $10M/year. He was thrilled about being able to drop his $40K/year Salesforce expense. One of his employees, let's call him Joe, a customer service person with some coding experience, used AI to create a CRM for the company that addressed most, if not all, of their needs.

He, the CEO, saw this as saving $40K per year. Until I asked simple questions:

What happens when Joe leaves?

When he goes on vacation?

If he has a health emergency?

The LLM isn't going to magically maintain the software, evolve, fix or support it.

What are you going to do?

The obvious conclusion was that he likely had to hire another person to work with Joe on this in-house CRM and, at a minimum, have redundant project ownership. Backup for development, maintenance and support.

The easy conclusion was: This will save him $40K a year until he decides to hire another person, at which point this "free" software will cost him $150K per year, $110K/year more than what he was paying Salesforce. If he does not hire a second person to work in the internal CRM, he might get lucky and things could be fine for a few years or he will have to face a crisis at some point when Joe is no longer around and nobody knows the first thing about his mini-CRM, not even the LLM.

I wonder how many people are falling for this trap these days. The allure of zero-cost software vs. the reality of introducing risk, technical debt and organizational risk.

To be sure, we are using LLM's extensively. However, until this is better understood, in the realm of software development it is constrained to what I would call "advanced auto complete" at the file level rather than anything resembling project-wide work. When we do write full applications, they are often relatively trivial internal tools that a person could complete in not much more than one week. In other words, easy to understand and code if another engineer had to jump in and none of these utilities are mission critical.