alt Hacker News> problems that require knowledge that simply doesn't exist yet.
Obtaining the knowledge is a process of trial and error, bruteforce, observation, search, etc. Humans follow that same process. Machines can do that too. If they discover or are taught the same heuristics humans use, and a computational capacity greater than that of all humans combined, they will outpace us in this endeavour.
> Until "AI" turns into general purpose robots that can develop new tools to explore the world, it is, in fact, pretty damned limited in what it can do without human help.
For sure, but this isn't a question of if, but when. LLMs are already rapidly accelerating the pace of every scientific field, so advancements in technology will start to compound.
> Biology is replete with examples. Computers "solve" protein folding [1], and midwits immediately leap to conclusions that drug development will also quickly fall. But we literally have no idea how most of biology works, and simply getting to the starting line for drug development problems is often 95% of the battle. Come talk to me when you've done a million experiments to find the fundamental knowledge that unlocks the pathway(s) we didn't know about that makes a drug discovery program possible in the first place [2].
I don't disagree with you. The amount of complexity in biology is incredible and due to its unpredictable squishy and noisy nature and the advanced machinery needed to analyze it with precision at the microscopic scale, vast amounts of data accumulation are still required to discover the useful patterns here, and the search space will be enormous. Still, it will be done. Here is some recent movement in that direction: https://www.anthropic.com/news/model-hardware-standard-resea...
> I am not pessimistic about humans running out of challenges. We'll just declare one class of problems "done" [3], and move on to the next frontier, as we always have.
This is fair, you're probably right about that. I think they'll be challenges for the AI to solve though, not humans as a collective. Hopefully we have something to contribute to that process, other than just expressing our desires.
> The problem with AI doomers is that they lack imagination that extends beyond computers, or perhaps more accurately, are so sophomoric in their thinking that they skip over the hard parts of any problem they don't fully understand. This stuff reminds me of the endless smartypants whinging about the end of human intelligence when chess machines started beating grandmasters. Chess was never really that great a measurement of human intellectual capacity, and we found new things to do with our big monkey brains.
I'm not an AI doomer, I despise the AI doomer movement and the incessant fearmongering, it's incredibly unhealthy and its only purpose is to perpetuate panic in the aim of maintaining and establishing power, and preventing others from obtaining power that threatens the status quo. What cultish idealogical humans will do with this technology if they're left unchecked with staggering power and no one to stop them is certainly a big concern though.
> Obtaining the knowledge is a process of trial and error, bruteforce, observation, search, etc. Humans follow that same process. Machines can do that too. If they discover or are taught the same heuristics humans use, and a computational capacity greater than that of all humans combined, they will outpace us in this endeavour.
Sure, you can reduce the 99.9% of research labor that matters to "a process of trial and error, bruteforce, observation, search, etc.", but that's like saying that nuclear fusion is only a few technical details away from implementation. We already know the theory!
The part where you're closest to being correct is "trial and error" -- it would be great if a robot existed that could do any experiment, tirelessly, with the mechanical fidelity, intelligence and creativity of a human. That robot does not exist. Moreover, the fundamental techniques to do the kinds of observation necessary to unlock the parts of science we don't know about do not exist. They must be invented. So now we have two problems. The problems are recursive and interlocking.
Biology and chemistry are the sciences I know best, so I will use those examples -- every major breakthrough of the last 50 years has involved invention of some fundamental new mode of observation, such as crystallography, NMR, mass spec, electron microscopy, various kinds of light microscopy, DNA sequencing, PCR, etc. Someone invents some innovative technique, and a wave of progress happens. Expert practitioners in in the lab are probably the second rate-limiting step, but the part that LLMs can do -- taking data and turning it into hypotheses -- is the part that carries the least value. Any postdoc has enough ideas to keep a lab going forever.
The thing you linked about Anthropic creating a "robot standard" for operation of lab tools is great for Anthropic, but that's about all. There's tons of lab automation tooling already. Having LLMs run the microscope is maybe a cool automation technique if you have the kinds of experiments that benefit from it, but those are rare, and they're still ultimately limited by people doing the upstream work.
AI will certainly help people be more efficient at their current scientific jobs, make better methodology more universal, etc., but suggesting that it will replace actual scientists is just science fiction.