logoalt Hacker News

runarberg • today at 12:12 AM • 0 replies • view on HN

OpenAI has not released the model for replication (not even to select few), they have not even shared the prompts. All we have is their results and their compute times (which may even be lies). This is the opposite of the scientific method. The scientific method asks for falsifiability and replication. If a hypothesis is not falsifiable, it is not scientific, if results cannot be replicated, they are not scientific. OpenAI fails on the second account here.

Not being scientific is excusable in some context. A high-school prodigy might stumble upon an unexpected result that they had a hunch about, but fail to make those results replicatable. Similarly a YouTuber with under a 1000 subscribers might propose an unfalsifiable hypothesis as entertainment. Being scientific takes skill and effort, and it is not free, so we can’t expect anybody to follow the standards. However OpenAI claims to be a trillion dollar company, and they do not have the same excuses as a small YouTuber or a high school student. OpenAI absolutely can afford to follow the standards, if they claim to be furthering the human understanding, then they should be scientific about it.