alt Hacker NewsI think it would be great if you could try models with lesser parameters that could fit on 6GB VRAM-ish, which could work for "gaming laptops" as well.
I think it would be great if you could try models with lesser parameters that could fit on 6GB VRAM-ish, which could work for "gaming laptops" as well.
There are options. If you have fast CPU RAM (ddr5) and PCIe bus you can run Qwen3.6-35b-a3b at good speeds+~100k context (I have a friend who is running this setup). If not, you're stuck with the much smaller models: Ling-3.0-tiny, Spark-X2.5-4B, LFM2.5 in 8b-a1b or 2.6b, and FrogNano-4B-2609 looks promising. But 6GB is a tough squeeze, 8gb is a lot cleaner, and if you have 12gb you can run strata like OP.