The issues in the article sound eerily familiar to anyone who has seen the passing unit tests that LLMs love to generate.