> It seems by sharing KV cache all questions will be visible

Yeah, this is partially why in my approach I've done this instead, and not used diffusion model:

1. Convert the state into one shared prompt.

2. Run that shared prompt through the model once.

3. Fork the model’s internal state once per question.

4. Add a different question to each fork.

5. Ask each fork for its next-token scores.

6. Calculate only 64 possible label scores—not the whole vocabulary.

Basically ... skip decode.

Won't be as fast as doing diffusion model parallel across a pile of questions at once, but:

a) let's you use pretty much any existing text model (with some modifications). I've got qwen3.6 moe and qwen3.8 flash next running, am getting gemma4 working now

b) the problem you identified

It's possible I'm getting high on my own supply and misunderstand entirely the whole thing, but it seems to work?

https://github.com/rdaum/eider/commit/9c2d5c049068c33da2a48b...

I don't have the chutzpah to go creating PRs for vLLM to do the same.