Danluu had an article circulating on HN a few days ago that apparently shows that yes, they can. He implemented Zstd and most models did it almost perfectly in most languages. Pandoc was a challenge they still cold not one shot though, which seems correct to me as the LLM would need proper guidance still for such a large enterprise.

> Do you really think current LLMs can implement a large application in C as securely as they can in e.g. Go?

That's not my experience: I've seen an LLM generate a C++ use-after-free (1.5 month ago).

Why not? The vulnerabilities you read about are due to oversights, not inherently lacking capabilities of the language. For every program exhibiting any given of such, you'll find thousands which don't make the same mistake at the same place. After all, most reported vulnerabilities are just a short patch away from being fixed.

Now human programmers might find it more difficult to get certain things right than others, but to a LLM only quantity of examples matters, no?