alt Hacker NewsFlide is extremely fast when one input carries many questions, as the shared input is computed only once. Even with a single question, it can still be up to 3x faster, depending on the model size and input length.
Flide is extremely fast when one input carries many questions, as the shared input is computed only once. Even with a single question, it can still be up to 3x faster, depending on the model size and input length.
Unfortunately, the reverse situation, where the same question is asked about many inputs, is much more useful in practice.