alt Hacker NewsWe don't vary our model quality with time of day or load (beyond negligible non-determinism). It's the same weights all day long with no quantization or other gimmicks. They can get slower under heavy load, though.
(I'm from OpenAI.)
Replies
Can you be more specific than this? does it vary in time from launch of a model to the next few months, beyond tinkering and optimization?
Specifically including routing (i.e. which model you route to based on load/ToD)?
PS - I appreciate you coming here and commenting!
I appreciate you taking the time to respond to these kinds of questions the last few days.
Hi Ted. I think that language models are great, and they’ve enabled me to do passion projects I never would have attempted before. I just want to say thanks.
Has this always been the case?
I believe you when you say you're not changing the model file loaded onto the H100s or whatever, but there's something going on, beyond just being slower, when the GPUs are heavily loaded.
Thanks for the response, I appreciate it. I do notice variation in quality throughout the day. I use it primarily for searching documentation since it’s faster than google in most case, often it is on point, but also it seems off at times, inaccurate or shallow maybe. In some cases I just end the session.