> The state is entirely internal to the network and disappears after a token is generated,

Yes and no, but mostly no, at least within a context window.

Mathematically, you could write a single step of LLM decode as a pure function from a list of past tokens to a predicted token (or a distribution over tokens, if you consider sampling separately).

But nobody actually implements this, because each token depends on state computed at past tokens in a way you can reuse.

So, in practice, inference computes a very rich vector of state- at each layer, for each token. And models do indeed use this to plan and track things over time (you can see this in interpretability results, e.g. with linear probes or natural language autoencoders).

> Stop the GPU, hit ctrl-c on the inference server, pull the power plug, cut the ethernet cable, send a kill signal, etc - any of these stop submitting new batches to the GPU and halt execution. That stops tokens from being generated. Stopping a "rogue" LLM is that easy. No input, no output.

This is also true about a human brain. My brain isn't going anywhere- it can't move by itself. It's also easy to kill (without the rest of my body, it dies in minutes!)

However, malicious human brains- especially powerful human brains, like leaders of countries- are often quite difficult to stop, because they're able to control systems that can see, speak, walk, run, fire a weapon, and so on.

One such system is the rest of the body, of course, but there are others (consider a UAV pilot, Perimetr, or a powerful leader who tells other humans what to do).

The brain being squishy doesn't make the thing easy to kill.