You really don't need an entire new inference engine for this. I modified vllm to work just fine for my decision model experiments.

It properly uses the single shared context and only recomputes the questions, or if the questions are longer, I let the questions go first and then the context that changes often can go after, and then the answer menu tokens... I got roughly the same speed increase with the right workloads that you're saying though.