Sometimes you want a decent model running in the background that doesn't take up all the VRAM.
Or maybe even to run parallel threads of the same model!
alt Hacker News
Or maybe even to run parallel threads of the same model!