alt Hacker NewsI ran into the second issue very frequently for a multi-agent system I created recently. Initially, Opus 5.5 filled out all sorts of places I implied it should use a call to judge LLM with extremely brittle regex and string equality checks.
I hadn't actually considered that it could be some sort of safety filter, but it makes me concerned that if that's the case, it may also be comparatively bad at actually writing the prompts for judge/advisor LLMs once it implements them.
A bigger issue was asking it to implement a simple change to start recording the CoT tokens in my logs for debugging purposes (since I'm running everything on open models) and it completely shut down, sort of like the old safety filters on GPT 3 and 4 models where the model has trouble even understanding what or why it's refusing.
Replies
bias against invoking LLMs is maybe a result of token/task benchmaxxing?
I think asking for the CoT is a trick used by distillers, so that might be the guard you are seeing. I remember reading that the big providers were starting to implement such protections to deter their competitors. Which makes sense, to a degree.
But like any such security measures, it has demonstrated side-effects that can be argued are worse for them than the original problem. Too much “protection” like this, and developers switch to providers that do not neuter their products.