What I find most interesting here isn’t necessarily that Claude can predict protein sequences, but what this tells us about how knowledge is acquired by large models.
The shuffled-sequence control is particularly compelling. It suggests that the models aren’t simply recalling amino acid frequencies or reproducing patterns from scientific literature. They’re capturing structural information embedded in the sequences themselves.
There’s an important distinction, though, between learning statistical representations of protein families and actually understanding the underlying biology. Next-amino-acid prediction is a fascinating proxy, but it doesn’t necessarily demonstrate that the model has learned the mechanisms governing protein folding or function.
I’d be particularly interested in seeing whether these capabilities transfer to genuinely novel protein families, rather than previously unseen members of known families. The post-cutoff experiment is a good step in that direction, although it still tests generalization within existing families.
Either way, this raises an interesting question about foundation models: how much latent scientific knowledge are we inadvertently creating simply by exposing them to sufficiently large and diverse datasets?
Perhaps we’re underestimating what can emerge from learning the statistical structure of the natural world.
I wonder how they would do on novel protein sequences?
I mean, that’s the holy grail, no? In fact, I’m genuinely surprised the other labs don’t train on protein sequences. It’s both an excellent source of high quality information and potential breakthroughs.