logoalt Hacker News

dev_hugepagesyesterday at 7:23 AM2 repliesview on HN

What parameter would you advise for min_p?


Replies

selfhoster11yesterday at 9:39 AM

Check out this’s recent HN submission: https://gist.github.com/Hellisotherpeople/71ba712f9f899adcb0...

Der_Einzigeyesterday at 4:45 PM

as temperature approaches infinity, min_p must approach 1 to stay coherent.

Assuming your temp is below 2, min_p of 0.1 is fine (and disable top_p and top_k). You can try 0.05 for more diversity.

Remember that subsequent methods are better, min_p is a mid-tier sampler that just happens to be the best implemented in most inference providers right now.

Also I'm the author of the "conspiracy against high temperature sampling" thing that selfhoster posted in the comments, so you can ask any questions about that piece you want.