Stealing Reasoning Traces from Proprietary LLM APIs

This is a neat attack against those encrypted reasoning blocks you get back from APIs like OpenAI and Gemini and Anthropic:

> We take a trace produced by a frontier model, replay it into a weaker sibling, jailbreak the weaker model, and recover the stronger model’s hidden reasoning in plaintext.

Should be easy for them to fix though: switch up the encryption key so it only works with the API for each specific model, rather than being shared across all of their models.

And indeed, the paper says it's been fixed by all three providers (though no news on how they fixed it):

> All model providers acknowledged the receipt of our report and subsequently we were unable to launch the same attacks.

Proprietary reasoning can be recovered from its encrypted traces. Anthropic, OpenAI, and Google return encrypted chain-of-thought blocks to clients that can be replayed across sessions, users, and models. We take a trace produced by a frontier model, replay it into a weaker sibling, jailbreak the weaker model, and recover the stronger model’s hidden reasoning in plaintext, without ever attacking the stronger model directly or triggering its anti-distillation safeguards.

[flagged]

[dead]

[flagged]

The real issue is that API providers log everything. OpenAI/Anthropic already capture full CoT traces in their logs — they just don't expose them. Distillation via API is just making explicit what they already have.

I am slowly turning around on the idea of opaque reasoning tokens.

In principle, yes, I want total control and visibility into the reasoning process.

In practice, I find that it takes up so much time to DIY reasoning agents that I can't spend much energy on the actual application.

The model providers have way more resources and talent to do this right and keep it right over time. I am willing to concede this moat to them if it means I can actually focus on the business.

The more I think about it, the less I care to see those tokens. It feels an awful lot like obsession over logging every trace item an application could produce. Useful in theory but a nasty garage stacked to the ceiling with useless shit otherwise. What nefarious things are we concerned with? That they burn too many tokens in the black box? I'll threaten to move to a different black box. There are always options in a market with this many participants and big players feel this pressure. They know there's a point at which being evil bastards is no longer profitable.

Spending time carefully designing tools and views that interact with the environment in clean ways is a much better investment than trying to own and control 100% of the reasoning process or models.