Is math or theoretical Computer Science not "real science"? Whatever one may think about the process, the results are real and are coming from LLMs.

If you are limiting yourself to modern whatever, you are not going to find that a lot of science that affects everyday lives. Between the first results and practical applications, it can take many years. There is a reason Nobel prize winners tend to be old.

Less modern AI (aka. machine learning) is used all the time in science. I don't know of any commercial breakthrough yet, but there are some promising developments, like in antibiotics. If it materializes, that would be a very big deal.

This post so perfectly captures the tragedy of how the term AI has lost its meaning to LLMs.

I think maths is really the only solid case so far. I do wonder whether the big early contribution to experimental science will be in speeding up the scientists conducting it, at least until some non-negligible boundary in simulation fidelity is crossed.

AI has an incredibly long history of contributing to science and engineering. Some highlights:

Gerard t' Hoofd (under Martinus Veldman) proved in 1971 that Yang-Mills gauge theories are renormalizable, using a rule-based AI (LISP-styled but manually written in assembly) called "Schoonschip" (I'm guessing called that because there were several failed attempts that went before and failed). This program was an AI in the sense of a search algorithm searching symbolic proofs. It got the 1999 Nobel prize in Physics.

This was hugely important because first, it showed that it was possible to make the Standard Model of physics and it also helped beyond that by showing for a lot of (at that point considered) impossible theories exactly how to make them work (meaning not just a proof that it could be done, but a long list of "if you have problem X, do Y" recipes). One of these became first the "electroweak unification", then the standard model.

Hao Wang proved that the logic of "Principia Mathematica" was actually correct using another symbolic proof generator. His efforts were also the start of many other such programs that contributed lots of stuff to mathematics that nobody sane was ever going to do, such as finding all the higher-order versions of the quadratic formula (there's a lot of them).

At Stanford university, tens of thousands of molecules were found in 1965 by having an AI, this one was what's called an "expert system", evaluate mass spectroscopy data and combine it with chemistry knowledge in the form of rules about what was possible or not possible and if only one option remained, spit it out. It almost doubled the sum total of our chemistry knowledge if counted by known molecular structures. It was later adapted to make mass spectroscopy useful in finding the composition of samples.

Then there's Alphafold, first a ConvNet (Resnet+Convnet to be exact), then switched to the transformer architecture, which gave use the folding of just about every protein we have ever found, and even more proteins that we've never even found. Nobel prize in Chemistry 2024.

There’s been a tendency since well before LLMs for AI demos to focus on “here’s a new problem we made up that it solves that you could imagine will soon transfer to an existing real problem” as opposed to “AI solves X”. This has been an ongoing challenge, it does cool stuff and looks so promising but has been very hard to harness for useful work. It’s still going to take time before the hype dies down and we see what’s left.

I don’t think it’s going away, LLMs and other ML stuff will no doubt remain part of people’s workflow, but I also see no evidence for superabundance or anything close to it.

LLMs are amazing for engineering work. They can automate design and testing of the entire spectrum.

Saw a post recently about how AI agents were used to check if papers had flaws in their conclusions and it found 15% did, that kind of auditing is already a massive massive advancement for science

this announcement is not regarding a large language model