logoalt Hacker News

Tade0 • today at 9:57 AM • 0 replies • view on HN

I was not aware of that, so I downloaded the latest release (which includes but doesn't document this flag), but performance unfortunately tanked in my case.

Even If I set the fit target to a value which without the cache would leave that VRAM unused, I get about a third of my usual token rate.