alt Hacker NewsIt is bad if this points to a general pattern of AI being better at finding than fixing vulnerabilities. The larger that ratio, the harder to secure all the things. The current stat from this experiment is 37,966 to 217. Lots of false positives, but not enough to change the big picture. I hope they didn't apply equivalent tokens to both sides, or AI may be better at demolition than construction.
Replies
So far the trend is pointing to AI being better at finding problems than designing software to avoid them.
Which I'm fine with really, but it means you need to be very careful letting it iterate on code until it finds no problems.
Coding is hard. I wouldn't be surprised if any patch they suggest came with new bugs, as it often happens with humans.
It'll never scew to far of as long as defence is a RL target. Better red team just means the blue team has stronger target environmenta to RL for.
once you have documented test for a bug AI is VERY good at fixing vulnerabilities. If there is a human there to look thru the fix code