alt Hacker NewsFair. But that 1/10 continues to get more and more impressive. Say, Claude 7 creates a new, brilliant scientific idea every 1 out of 1000 times. Anthropic reports "Claude figured out how to tie general relativity with quantum mechanics." Would you hand-wave it away saying that it's cherry-picked?
Replies
> Say, Claude 7 creates a new, brilliant scientific idea every 1 out of 1000 times
Sadly, general public (us) is never seeing that model
It wouldn't surprise me that a company with an effectively unlimited budget could fund enough researchers to solve breakthrough problems, all while using LLM's (which are excellent research and exploration tools!) and then claim that the LLM found the breakthrough. The amount of money Anthropic has to play with is >10,000x what entire fields of research have, I think people don't appreciate how few resources we spend to support people working on hard problems that don't have clear commercial applications.
Very similar story with security research, LLM's are a super useful tool while hunting vulnerabilities, but it turns out when the entire software industry starts throwing tens of billions of dollars at vuln. research, a lot of stuff gets unearthed, something security people have been insisting on for years and complaining that their work is underfunded and under-resourced.
> Say, Claude 7 creates a new, brilliant scientific idea every 1 out of 1000 times.
I would not hand wave that away - but Claude 5 feels closer to Claude 1 than the hypothetical Claude 7 you propose. I do not believe one can extrapolate LLMs that far ahead despite the very substantial progress so far.