> Fable 5.2 may behave well because it mistakenly assumes it is being tested for safety. As models become smarter, they may learn that most interactions are not safety tests. If that mistaken assumption is what keeps Fable 5.2 behaving well, future versions may not behave as safely.
It’s cryptic lesswrongese. I hate the writing style so much.
That sentence is very straightforward. How would you rewrite it?
There are some reasons for the writing style being that way. Even today, when we have AI systems we can directly observe, many people make mistakes in thinking about them. Back when this style of writing developed, mistakes were even more common. Using this particular and difficult style was historically (and to a lesser degree, is still today) a hard to fake signal that your thinking could be trusted to avoid some pitfalls.