logoalt Hacker News

DamnInteresting • today at 2:09 AM • 1 reply • view on HN

> The mathocalypse didn't happen because they found better training data somewhere.

There are reasons to suspect that OpenAI may have intentionally or unintentionally trained on chat logs from mathematicians--mathematicians who were actively using OpenAI to try to to solve the same problems themselves.[1] If those mathematicians were on the right track, and getting close to a solution, that could be classified as finding "better training data somewhere."

[1] https://cybernews.com/ai-news/openai-accused-stealing-anothe...


Replies

Shaanveer • today at 5:42 AM

sorry but given how post-training actually works, there's essentially no direct way in which literal PII-scrubbed data made its way into a production oai datamix for RL warmup purposes in 2026 to the extent this data has influence, it's indirect/at the level of process decisions. what i am trying to say more specifically is, the folk model some people seem to still have about ai companies "directly training on user transcripts" in this naive fashion is firstly wrong, and secondarily, would be counterproductive as a strategy for continued progress "transcript shows human trying to do an exotic thing evals don't currently cover -> build synth constructions that broadly cover its genre of problem class -> RL on the verifier set designed to cover the failure" that's the loop... the """flywheel""" is... diagnostics! wow! and so this kind of process improvement driven use of data has an obvious diffuse causal influence on what happens during training, right? but ~none of that process influence looks like "specific mathematician's specific intuitions leak into the training data".

➕ show 1 reply