I would assume that you could get by with a very small and fast LLM to do the question generation. You’re not asking the model for very much. From what I can tell, Jev is adding the quantitative probability which makes the score not simply a sum of ones and zeros against the questions.