logoalt Hacker News

frunkpyesterday at 7:46 PM0 repliesview on HN

When I saw "hidden reasoning", it reminded me of diffusion models: generating a block spans many steps (with remasking), which hide the reasoning that led to the block.

I had not heard of looped transformers, but the engineering behind the number of loops per token / halting feels like trying to apply a diffusion process to a transformer while keeping the auto-regressive feature.