alt Hacker NewsWait, what has OpenAI done that is not “consistent with the scientific method”?
https://www.sciencebuddies.org/science-fair-projects/science...
Do other researchers or research institutions “pay mathematicians (other than those in their employ) to go over these papers” or do they simply publish their work for review as part of producing “reproducible results”?
AI may be taking over Mathematicians’ jobs, along with everyone else’s, and it’s OK to hate that, but the scientific method says nothing about that, or hiring experts, or ulterior motives, or “being filthy rich.”
OpenAI has not released the model for replication (not even to select few), they have not even shared the prompts. All we have is their results and their compute times (which may even be lies). This is the opposite of the scientific method. The scientific method asks for falsifiability and replication. If a hypothesis is not falsifiable, it is not scientific, if results cannot be replicated, they are not scientific. OpenAI fails on the second account here.
Not being scientific is excusable in some context. A high-school prodigy might stumble upon an unexpected result that they had a hunch about, but fail to make those results replicatable. Similarly a YouTuber with under a 1000 subscribers might propose an unfalsifiable hypothesis as entertainment. Being scientific takes skill and effort, and it is not free, so we can’t expect anybody to follow the standards. However OpenAI claims to be a trillion dollar company, and they do not have the same excuses as a small YouTuber or a high school student. OpenAI absolutely can afford to follow the standards, if they claim to be furthering the human understanding, then they should be scientific about it.