alt Hacker NewsAnd if three of their 700 PRs were retracted within a day because of unresolvable bugs? Clearly it's not all verified.
Not to mention the math paper (on HN yesterday) which pointed out that OpenAI might have verified the wrong thing, in their Navier Stokes proof.
It's really not as cut and dried as you (and software/AI folks more generally) think it is.
PS: If a human mathematician had to retract three of their papers a day after posting publicly, they'd lose all credibility and their mathematical career would be all but finished. That social incentive structure is the field's immune system against slop. You're basically asking them to turn off their immune system, and for unclear gains (other than OpenAI's grandstanding).
If a human mathematician had to retract three of their papers a day after posting publicly, they'd lose all credibility and their mathematical career would be all but finished.
If a human published 700 papers and only 3 (or 30) ended up having significant errors, I'd call that a pretty good batting average, considering that the typical rate of errors may be around a third (https://lamport.azurewebsites.net/pubs/statistics.pdf). But for some reason people hold AI output to an absurd standard where if it's not 100% perfect then it's useless.
You're basically asking them to turn off their immune system
I'm not asking "them" to do anything. They can do whatever they want, including rejecting obviously useful tools. But then they shouldn't be surprised when they're quickly surpassed by others who don't share their ideological blinders.