alt Hacker NewsI've seen tests doing:
a = 1
assert a == 1
// many lines here where a is never used
assert a == 1
Yes AI test cases are awesome until you read what it's doing.
Replies
alecbz • yesterday at 7:24 PM
It's good at writing/updating tedious test cases and fixtures when you're directing it more closely. But yes, it's not as great at coming up with what to test in the first place.
gzread • yesterday at 12:10 PM
I write assert(a==1) right before the line where a is assumed to be 1 (to skip a division by a) even if I know it's 1. Especially if I know it's 1!
To be fair, many human tests I've read do similar.
Especially when folks are trying to push % based test metrics and have types ( and thus they tests assert types where the types can't really be wrong ).
I use AI to write tests. Many of them the e2e fell into the pointless niche, but I was able to scope my API tests well enough to get very high hit rate.
The value of said API tests aren't unlimited. If I had to hand roll them, I'm not sure I would have written as many, but they test a multitude of 400, 401, 402, 403, and 404s, and the tests themselves have absolutely caught issues such as validator not mounting correctly, or the wrong error status code due to check ordering.