logoalt Hacker News

programjames • yesterday at 10:12 PM • 1 reply • view on HN

Most people who write ML papers are at the stage of alchemy—try a bunch of things and see if you can create gold out of lead. Usually their papers are not worth reading, because even if they do something right, they do not know what made it right, and it cannot lead to further or generalizable insights.

A few ML researchers do actually have a lot of this theoretical knowledge. You can see this in papers like Variational Diffusion Models, Continuous Normalizing Flows, <?> Delta Attention. These papers are usually also better written, because the authors understand why things work so they can deprioritize irrelevant details.


Replies

nsagent • yesterday at 11:32 PM

Ali Rahimi's NIPS 2017 Test of Time Award speech lays out that case especially well:

https://youtube.com/watch?v=x7psGHgatGM

I presume your comment is in reference to this speech, as he famously states: "Machine Learning has become alchemy" in it. For anyone who has some time and hasn't watched it, I'd highly recommend it.