logoalt Hacker News

spijdar • yesterday at 10:14 PM • 0 replies • view on HN

It's worth pointing out that merely having the chain-of-thought in latent space isn't by itself "The Most Forbidden Technique". Rather, it's doing what the authors propose as the next step in the paper:

  Hybrid generation can also make post-training more efficient. Because latent thoughts remain auditable during rollout, an external critic can assign step-level credit to latent spans, branch promising trajectories, or terminate ones whose latent reasoning has already gone wrong, before any text or tool call is produced.
Unless I'm misunderstanding, they're suggesting RL training against the CoT. Which, uh, that is The Most Forbidden Technique, as I understand it, when combined with the latent reasoning space. It allows for the model to "learn"/optimize for malicious behaviors that are just far enough away from the quantized token they're auditing against.