What would be the incentive for someone to do this for real?

We all have access to SOTA LLMs. If I want a "clean room" implementation of some OSS library, and I can choose between paying a third party to run a script to have AI rebuild the whole library for me and just asking Claude to generate the bits of the library I need, why would I choose to pay?

I think this argument applies to most straightforward "AI generated product" business ideas. Any dev can access a SOTA coding model for $20p/m. The value-add isn't "we used AI to do the thing fast", it's the wrapping around it.

Maybe in this case the "wrapping" is that some other company is taking on the legal risk?

What do you mean nobody has done it?

It's an inevitable outcome of automatic code generation that people will do this all the time without thinking about it.

Example: you want a feature in your project, and you know this github repo implements it, so you tell an AI agent to implement the feature and link to the github repo just for reference.

You didn't tell the agent to maliciously reimplement it, but the end result might be the same - you just did it earnestly.

There's a lot of things you could do to be malicious towards other people with minimal effort, yet strangely few people do it. Virtually everyone has morals, and most people's are quite compatible with society (hence we have a society) even if small perturbations in foundational morals sometimes lead to seemingly large discrepancies in resultant actions

You need the right kind of person, in the right life circumstances, to have this idea before it happens for real. By having publicity, it becomes vastly more likely that it finds someone who meets the former two criteria, like how it works with other crime (https://en.wikipedia.org/wiki/Copycat_crime). So thanks, Malus :P

At some level people are already doing this through LLMs. But large orgs are extremely risk averse to do such things. There’s a reason why we have “security audits” and “compliance certifications”. It’s not like organizations are not capable of securing or standardizing their systems, just they do want to point fingers to somebody when legal proceedings happens.

The bottleneck is trust and security. I'd rather defenestrate 3rd party libraries with a local instance of copilot than send all my secret sauce to some cloud/SaaS system.

Put differently, this system already exists and is in heavy use today.

> why hasn't anyone done this for real?

WDYM? LLMs are essentially this.

>why hasn't anyone done this for real?

because LLMs can't program anything of non-trivial complexity despite the persistent delusions from its advocates, same reason the lovers of OSS haven't magically fixed every bug in open source software.