alt Hacker NewsIt runs very quickly on my RTX 5090 fwiw. Whole thing is loading entirely into vRAM with a ~130k context size (the max) fitting as well.
It runs very quickly on my RTX 5090 fwiw. Whole thing is loading entirely into vRAM with a ~130k context size (the max) fitting as well.
That's a $5k 32GB card for anyone who doesn't know all these off the top of their heads (like myself).