alt Hacker NewsFirst through as I read this: Microservices.
> This essay introduces the concept of decoherence: the accumulation of locally valid but mutually incompatible or redundant models within a system, such that the cost of change is dominated by translating among those models and preventing unintended side-effects.
But then famous last words:
> The good news is that this rules out the specter of runaway recursive self-improvement, the intelligence explosion that goes parabolic before anyone can react, as decoherence both prevents AI from improving in a bubble and limits the rate at which it can improve outside of it.
After the OpenAI/Huggingface and similar I am not so sure. Even if that was a publicity stunt of sorts, I think it shows they can go off the rails. Today it is hacking, but tomorrow it may be training a smarter model to join their cohort, without us being involved.
LLMs can be extremely dangerous without being capable of achieving super intelligence in a vacuum.
If I hooked up a random number generator such that, if it landed on a specific number between 1 and a trillion, it would launch a nuke, that would be incredibly dumb and incredibly dangerous.
Even more reason to slow down the slop code and stop attempting to achieve super intelligence when it appears that it will not be possible with this technology. The process is dangerous, and the doom race/prisoner’s dilemma logic is faulty.