logoalt Hacker News

mattstir • yesterday at 8:44 AM • 1 reply • view on HN

The statement appears to be referencing Astra's supposed recurrent depth and how that causes reduced visibility into chain-of-thought reasoning. Astra's own system card describes tests where it's asked to solve challenging math problems while internally thinking about something entirely unrelated, which it's significantly more capable of than previous models (~60% vs ~16% of the time for Sol). That seems to point to reduced efficacy of chain-of-thought monitoring, but OpenAI's public statements basically boil down to "yeah but we haven't been able to catch it doing that" which isn't exactly reassuring if CoT monitoring is one of your main safety guardrails.


Replies

WarmWash • yesterday at 1:56 PM

Unfortunately, this reduced insight seems to also give Astra incredible "intelligence-per-token".

The worst case scenario is that the easiest path forward is one where we lose sight of internal thought.