No it doesn't. It produces working, but sloppy code, and it's able to do so because of deterministic test suites that run quickly and give a reliable score of the result, letting an agentic loop iteratively improve its output. But we have no test suites that answer "is this code maintainable" or "does this feature make sense in the context of our business", so AI still fails those tests.

If we had a way to write a test suite for English that could reliably answer "is this sentence useful to my target audience" or "does this paragraph make any goddamn sense", you might be able to run some sort of agentic loop to approach good writing, but so far we don't.

No, it also produces impressive looking code that is actually not that great. It's exactly the same.

You can verify that code is good if it works - it might be ugly or hard to understand, but if it completes the goal and doesn’t crash then it’s good. You can’t really do this with prose though because the text is the thing itself - it needs a human to score it (perhaps another way would be to have other agents ask it questions about something they don’t know, and see if the agents can discern the answers from the output - but that’s still not going to determine if it’s good for human consumption).

Sometimes it doesn't, but it's tough to see because of the volume of what you have to wade through

I think it is more about comprehension. It reveals that AI does not actually understand what it does. It gets code right mostly because it is probabilistic and has seen enough sound precedents to land in something that works, but it does not come from an internal comprehension.

Unlike human languages, computer languages have tightly-specified boundaries so LLMs do a lot better. See also math.