One of my reports just sent me a giant design doc that Claude enthusiastically generated packed with plausible looking technical detail. Unfortunately the problem it's trying to solve is completely misguided and we shouldn't be doing it at all. So I'd say as answer to the question posed by this title: a while.
That's because you're still in the loop to point out it is trying to solve a misguided problem. Once it is LLMs all the way up - or at least far enough up so that the layers above it don't have the required technical knowledge to deduce whether the machines are following the correct track - they'll be solving problems 'till the cows come home no matter whether they're worth solving.
Yeah but these models are the worst they’ll ever be! They’ll get better! You need to give it better context! Did you try multiple agents?
Sarcasm aside, I am just so tired.
Your point of view is flawed. You should take the ratio of good / bad AI plans activity and compare it with the same ratio by human work.
We should trust your assessment.. why?
Look, you might be the most knowledgeable person on this planet but we cannot verify. Your “report” might be brilliant and you are just a dinosaur. I mean, it’s impossible to tell.
Also, deciding on whether something is a good idea for the business is not a thing LLMs are currently trained for. We are busy automating development which will in turn accelerate all other automation. Deciding whether some line of thinking is a good idea does not strike me as particularly outside the range of capabilities we are witnessing them exhibiting today.
>Unfortunately the problem it's trying to solve is completely misguided and we shouldn't be doing it at all
Isn't that like 90% of project plans by pre-LLM managers too?