alt Hacker NewsI've been trying the Q4_K_M version, and sometimes it gets stuck in a loop. Gemma 4 doesn’t have this issue.
Replies
Readerium • today at 1:10 AM
perhaps increasing repitition_penalty might be helpful
I've been trying the Q4_K_M version, and sometimes it gets stuck in a loop. Gemma 4 doesn’t have this issue.
perhaps increasing repitition_penalty might be helpful
This has happened before with quantizations and other backends (ones not used by the research lab). Give it a week, download latest versions of everything, and try again.