logoalt Hacker News

seizethecheese • today at 6:35 PM • 1 reply • view on HN

Blog posts claiming differences but without any statistics to show whether the differences are noise are a big pet peeve.

(For what it's worth, I asked an agent to compute p-vals based on available information for all the stats in this blog post. More than half are significant but a lot are noise. Not going to post the results because I haven't validated, and the agent made a lot of assumptions. The author should update the post.)


Replies

LPisGood • today at 6:43 PM

If you calculate enough different p values you will get a lot that are significant and a lot that are noise.