I mean I haven't checked but I'd guess it'd be possible to save a kv cache to an ssd or ram and then pull it back into vram when needed.