logoalt Hacker News

t0mek • today at 2:15 PM • 3 replies • view on HN

In "Empire of AI", the author mentions that when OpenAI included public GitHub repositories in its training data, the resulting model achieved better results even in fields unrelated to programming.

Aside from the obvious applications in biochemistry, could protein sequences also be used as a new training data source to improve overall model quality? If that's true, it's quite fascinating that we can take highly complex data we don't fully understand and use it to improve LLM training, simply because it contains so many significant structures and relationships.


Replies

phkahler • today at 2:36 PM

Software is used in all fields, and github issues and PRs are going to contain detailed reasoning about all sorts of things when adding features or fixing bugs.

➕ show 1 reply
origini • today at 4:45 PM

We have seen it happening in math and physics.

We have seen that LLMs are quite good in reading binary and changing binary directly.

Could be possible.