depends on how they're spliced, they have to be tuned for each configuration as there is no silver bullet. It also requires customizations to sglang/vllm to get it working in the first place so it's definitely pretty involved. However, it's still cheaper than obtaining more gpus.
depends on how they're spliced, they have to be tuned for each configuration as there is no silver bullet. It also requires customizations to sglang/vllm to get it working in the first place so it's definitely pretty involved. However, it's still cheaper than obtaining more gpus.