alt Hacker NewsSo far this year, of the defects severe enough to warrant a formal RCA process that I’ve dealt with, “AI generated tests asserted that the defective behavior was correct,” was a contributing factor in every single one.
So far this year, of the defects severe enough to warrant a formal RCA process that I’ve dealt with, “AI generated tests asserted that the defective behavior was correct,” was a contributing factor in every single one.
I haven’t had this happen yet, but I was just thinking about this last night. I would be very interested to see if exercising TDD, as difficult as that practice is to actually do, would have a meaningful positive impact on improving this phenomenon. Usually agents model what most humans do, writing tests to pass and assert that the code you just wrote works as you wrote it.