alt Hacker NewsThe major concern with looped transformers is that makes it more difficult to monitor model alignment. When more processing occurs within latent space without outputting text, that means less effective, frequent chain-of-thought monitoring, and the potential for greater un-monitored latent-space shenanigan.
Replies
technotony • yesterday at 8:54 PM
I'm not sure. That paper from anthropic talked about monitoring j space, presumably those same techniques would work here?
➕ show 1 reply
This is silly, the entire reason why chain of thought even exists is to let the LLM "think independently" instead of minimizing the deviation from the supervised training sample. It's an intentional scratch pad for intermediate data. The loose monitoring is kind of the entire point.