logoalt Hacker News

Imnimoyesterday at 6:18 PM2 repliesview on HN

>we did not read any private chats

The question I am interested in is not "did we read private chats", but "was this new model trained using any of Tristan and Levent's chats, regardless of whether they were marked private". Can you comment on that?


Replies

tedsandersyesterday at 8:57 PM

If they opted out of training, then we definitely did not train on them.

If they did not opt out, then I don't personally know if training signals came from their chats, and I don't think we'd be able to tell without their cooperation in identifying them. And even if signals were trained on in some manner, I highly doubt it made a difference to a problem as challenging as the NS proof.

Reasons for my doubt:

- I know most of our training recipes

- Our model's proof is very different from theirs

- The proof took a tremendous amount of tokens to derive (it wasn't a recall/lookup type question)

- This unreleased model has beastly performance on many unsolved math problems, not just the Euler solution

I acknowledge that this requires trust, and if you think we'd lie shamelessly about this stuff, then nothing we say can really help our case here.

Reminds me a bit of the Frontier Math fiasco, where people accused us of training on the eval set (we didn't), but it's hard to convince someone if they think you're lying.

If you're convinced we lie and cheat, then nothing I say may help. But if you're not sure, then hopefully providing my perspective is helpful.

show 5 replies
moralestapiayesterday at 8:32 PM

>Can you comment on that?

No answer is also an answer.

He's a human, like everybody else. Mostly a bunch of hungry animals looking to put bread in our mouths. It's rarely ever something a bit more sophisticated than that.

show 1 reply