Well if you consider the outputs of an LLM "thoughts", then they literally already do that, auto-regressively.

If your argument is that they're not "their" thoughts, I would agree with that. But I'd also say that no one develops their own thoughts. We're all just developing thoughts from the knowledge bank we have from our experiences in the same way that AI is drawing from its weights.