> The thing is, you start to run out of that as you lean in. Tricky.

I think it depends how you lean in. I generated a few thousand lines of Rust code today. It was actually pretty high quality. On my initial review, I only identified one change I wanted to make.

Most functions were pretty short and well factored. I wonder if using Rust as a target helps, because (1) the training data doesn't include all the terrible Javascript and Python etc. found on the internet, and (2) Rust and many of its libraries encourage good architecture - code factored into traits, functional pipelines, a powerful type system, etc.

LLMs trained on bad code for mediocre languages naturally write corresponding quality code.

A big part of it still boils down to prompting - what you're asking it to do and how much freedom you give them. If you let them make all the architectural decisions, you're likely to end up with issues.