alt Hacker NewsI'm running a Ryzen 7 8745HS with Radeon 780M (27GB RAM), using Ollama's Vulkan backend.
I get ~45 tokens/s generation and ~680 tokens/s prompt processing.
The problem is ollama is not caching the prompt, I get -> forcing full prompt re-processing due to lack of cache data
So this wouldn't work in its current state in an agentic loop.
Replies
Tostino • today at 7:35 AM
Ollama is generally just not great software. Use something else and you likely won't have that issue.
Friends don't let friends use Ollama: https://sleepingrobots.com/dreams/stop-using-ollama/