alt Hacker NewsMore questions about whether researchers can trust OpenAI with unpublished math
Comments
short answer seems to be "no"
the fix is boring and known: BIG-bench shipped a canary GUID for exactly this, and you publish your decontam n-gram threshold (gpt-3 used 13-grams). no threshold disclosed, no claim.
Feeding documents into a copy machine and getting progressively angrier and more confused as it prints out copies of them. Incandescent with rage I scribble “WHY IS IT DOING THIS?” on a scrap of paper and put it in the scanning bed
> The Wednesday evening statement from OpenAI was more emphatic: “We can say categorically that it is impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training.”
> The statement added, “After investigating, we can say with full confidence that no user inputs past July 3rd could have influenced this system in any way.”
https://www.nytimes.com/2026/09/10/science/tristan-buckmaste...
I'm confused by a lot of this discourse...
What have they done to show they can be trusted?
Forget researchers you as a business are putting in your business optimizations, your processes in order to train it so that Ai can then give that information to your competitors once incorporated into its training set. You are literally training your competitors.
Astra is strange. I asked it to design a treehouse and it just stopped every couple of minutes telling me what it still had left to do. After dozens of continue prompts it finally gave me a structure that would work but it was 10x more wood than I needed. I think the key mistake I made was asking it to “approve” the design for building. As soon as I asked that of it, it started getting “scared” and “apprehensive” and wouldn’t complete what I asked of it.
Its the "with unpublished math" that I have a problem with.
These companies are effectively high-tech plagiarism factories run by CEOs who are competing viciously. Look at their past actions. No, of course you can't!
people seem to miss tge point of this. The problem isn't about credit, its about portraying these models as more competant than they really are. It fuels idiotic statements like jensen huangs recent "agi achieved" statement, which fuels an already dangerous financial fire.
Big Tech will slurp up every piece of data it can about you and sell it to anyone it can, all to make you the target of someone else's goals, whether that is an advertiser, an employer, law enforcement, a stalker, or the government.
You will not be able to opt out unless you completely isolate yourself from society, tough shit.
Worth noting both ChatGPT and Claude have per-conversation modes (temporary/incognito chat) that are excluded from training.
What happens if you use a different harness?? Does opting out online suffice?
One question has been nagging me for this situation. Levent Alpoge works at Anthropic and would presumably have some knowledge of "how the sausage is made" and I would hope he would be aware that his collaborator was utilizing LLMs in some capacity for their joint work. Would he not have guided him otherwise if it were an open secret that this kind of thing was a possibility?
why would anyone trust anything from a company built on stolen data that spews hyberbolic, misleading claims.
Some mathematicians I know who've been following this have realized that they'd all gotten some emails from people they now know to be affiliated with OpenAI/Anthropic asking questions about their research in a way that seemed like scooping attempts.
Also, a lot of my mathematicians buddies have reported students basically asking if it's worth ever doing grad school for pure math, and even very motivated students are looking for other options now. It's not because they aren't passionate about it, it's that they don't want to work for another half decade or more just to have to start their careers all over.
All of this so that OpenAI and Anthropic can get into math result dick measuring to gas up their IPOs. Sickening.
OpenAI is showing the world why they shouldn't trust AI hosted on some cloud somewhere.
If they're stealing math proofs to advertise their models, who's to say they won't steal your businesses IP to gain a competitive advantage?
They're not to be trusted with your data. I can't believe how short-sighted this is, they got a quick PR win at the expense of a much larger trust problem.
I wouldn't trust cloud AI at all at this point. Get an open Chinese model and host it yourself somewhere. The initial costs might be higher, but you'll break even pretty quickly and nobody will be able to steal your innovations.
This is American AI companies committing suicide.
Gonna need grants for local models. Its happening. OpenAI and Anthropic models are powerful but are rapidly approaching the good ol trust thermocline.
I hate to be that dinosaur, but this is exactly what I’ve been warning about since XaaS began taking off in the mid-oughts: any provider you use can and will use your data for their own benefit regardless of any contracts or safeguards in place, especially if the benefits outweigh the consequences.
Honestly, I’m surprised it took this long for some company to really go all the way, though. OpenAI really making it transparently clear that they can and will do whatever they want with the data you provide them, contracts or settings be damned. Completely untrustworthy as an entity, full stop.
Of course, I’m also too jaded to think this will change anything. Folks will move to Anthropic, or Gemini, or Grok, or some other hosted model on a pubCSP managing the harness and logs for them, and then do another shocked-Pikachu face when it happens again.
If you aren’t running workloads on infrastructure you own, then your privacy, security, and general outcomes are at the sole whims of the hosting provider - who can and will fuck you over the exact second it’s more beneficial for them to do so than the loss of trust incurred.
The oxymoron of OpenAI in its name and its actions should give the collaborator all he/she needs to know.
Imagine what happens when you use a Chinese model. Seriously, just think about how much more control and visibility you have w US companies compared to CCP-controlled ones.
I wonder if the company that stole most of their training data and is led by a liar stole unpublished academic work and lied about it. What a mystery
And I have been proclaiming a cry of “your data for analytical purposes is being stolen” (you can’t opt out of analytical purposes) and people perhaps astroturfers straw man back to “just turn off training bro”.
Yeah. Remember yall: you CAN NOT opt out of analytical purposes. And you also cannot get a guarantee that it doesn’t give them your data to steal for their business.
>AI is stealing human discovery.
AI is not stealing human discovery, OpenAI is.
>Trust
No you cant do that lmao.
Prompts are handled by the service itself, meaning it's used, absolutely anything passing there is recorded, why wouldn't it, the entire premise of those companies is to train on data which they stole initially.
Are we back to the era where people blindly trust product TOS instead of actual cryptography, have we forgotten already the thousand of fines Google, Microsoft, Apple and practically all top companies got for breaching their own ToS and the law?
Common, on HN at least I would have thought that everyone assume that anything arriving on a server in PLAINTEXT is recorded (thus used later)?
Let's not forget that at any moment, OpenAI/Anthropic/Google... could be providing stronger privacy guarantees by having proper attestation with e2e, they have the budget, solid engineers, why isn't it done? Answer is pretty simple imo.
The real annoying thing it seems is mostly that openai is presumably doing this for internal reasons and this marginally increases the cost to users with no real gain.
It would be one thing to gain from it but removing prestige wins from customers AND reducing compute support just feels like being ultra mean if you zoom out.
If this was racing to cure cancer ahead of researchers we wouldn't be writing about this on HN.
Silicon Valley is flying head first into a FAFO train wreck on trust with everyone else.
OpenAI is firmly earning a reputation as a company where people just assume they’re up to no good. Rightly or wrongly that’s a terrible place to be.
The AI industrial complex in general is finding out hard what happens on the data center side when you get arrogant with local communities. Politicians have seen the polling numbers and folks you wouldn’t expect are running to the front of the crowd with pitch forks in hand.
Silicon Valley has totally lost the narrative here, but also lacks the self awareness to grasp how bad things are and will get and what that means for their own business viability.
This goes beyond math. The external appearance is that OpenAI negligently or intentionally used user data against a user’s interests in a potentially career-altering way… for the marketing bump of an AI proof. The fact that it probably wasn’t intentional is irrelevant. Trust has been lost and this will ripple into communities where no one has heard of Navier Stokes.
[flagged]
[flagged]
[flagged]
[dead]
[dead]
[flagged]
[dead]
[dead]
[dead]
[dead]
TL/DR: Mathematician opted out of training on 29-JUN and asked OpenAI whether they trained on his data and was told that it "did not happen" but it clearly did.
Trust and OpenAI never go together in the same sentence. The answer is always no.
But China steals our AI!!!!!!
I think this is mathematicians coming to grips with the fact that AI is surpassing them
we will all have this moment soon enough, and it will change how we think about intelligence, identity and value
Hopefully OpenAI can solve good problems where no one can make a claim that they stole their idea where the methodologies used and the techniques used are so out of the ordinary that the achievement is respected. Furthermore they should work on new novel solution on the old problems to lay these issues rest, thus by improving their models they can avoid issues of academics accusing them of using their work additionally the academics should also demonstrate their unpublished work is significant enough to have solved the problem . This is a bit subjective but it is also objective for the person with domain knowledge.
This is just mental illness at this point. I don't blame the mathematicians that have found a way to get attention from the mainstream press for once, but we should not fall for it here.
1. No one but OpenAI has produced a proof of NS so these accusations of plagiarism are pretty embarrassing. It reminds me of the line from the Social Network: "If they invented Facebook then why didn't they invent Facebook?". If these people proved NS before OpenAI where is their proof?
2. If they plagiarised Andreas Thom then why was his initial response to praise the proof and talk about how different it was from his own attempt? It's only now that it is clear that no one bothers checking these things that suddenly his story changes.
But who are you going to believe? Multiple independent academic researchers or the CEO who was fired two years ago for gross dishonesty?