It might be a mistake to assume tomorrow’s training looks like today’s. Unsupervised learning is a thing and a very hot research topic, precisely because it avoids some of today’s big problems with acquiring the vast amounts of training data necessary.

Maybe there aren’t that many new/necessary ideas that can be mined from the fundamental building blocks of software development (languages, syntax, runtimes, static analyses, type checking, etc). Maybe people will continue to innovate by instructing models to build novel things out of those building blocks? Perhaps things we would not have thought of building before due to the effort required without LLM assistance.

It's the opposite. The less competent the average developer the more valuable coding LLMs become (as the only way for those bad developers to generate ok code). Eliminate the good developers and even bad coding LLMs become valuable.

I don't expect them to realise that until some time after it actually happens. When it remains a future hypothetical, it won't be accounted for.

https://en.wikipedia.org/wiki/Profession_(novella)

fwiw 90% of software is reinventing the wheel. 80% of devs have an itch to "rewrite from scratch".

AI will deduplicate all of this

Why would there be a lack of original ideas? People who are born to code so to speak will do it. Information wants to be free as the saying goes. It only takes one time for an innovation for it to be to copied everywhere.

We don’t need the same volume of developers to have the same or faster speed of innovation.

And conversely if there is stagnation there is a capital opportunity to out compete it and so there will be a human desire to do the work.

Tl;Dr. People like doing stuff and achieving. They will continue to do stuff.

ps it’s too much to claim other people don’t experience creative ideas using AI. You don’t really know that’s true. It hasn’t been my experience as I have had the capability and capacity to complete ideas on my back burner for decades and move onto the next thing.

Innovation is irrelevant to pushing up this quarter's numbers. No one actually values unique and novel ideas. The only thing that matters is shipping something right now that can make an impact on this quarter's numbers.

Who cares if it's derivative slop or a straight up bootleg of something else so long as the number goes up

It isn’t about training anymore. It is about harnesses.

Just look at new math proofs that will come out, as one example. Exploration vs Exploitation is a thing in AI but you seem to think that human creativity can’t be surpassed by harnesses and prompts like “generate 100 types of possible…”

You’re wrong. What you call creativity is often a manual application of a simple self-prompt that people do.

One can have a loop where AI generates new ideas, rejects some and ranks the rest, then prioritizes. Then spawns workloads and sandboxes to try out and test the most highly ranked ideas. Finally it accretes knowledge into a relational database.

Germans also underestimated USA in WW2, saying their soldiers were superior, and USA just had technology — but USA out produced tanks and machinery and won the war through sheer automation, even if its soldiers were just regular joes and not elite troops.

Back then it was mechanized divisions. Now it is mechanized intelligence.

While Stalin said: Quantity has a quality all its own.

[dead]

Technically all the problems that almost any given business needs to be solved today has already been solved umpteen times over the years. There are no new problems that can't be solved by porting and/or combining old solutions.