alt Hacker Newsyeah but using the rule of thumb (I see floating in the Internet) which is sqrt(#params * #active params) which would give sqrt(284B*13B) = 60B, so deepseek should perform better.
yeah but using the rule of thumb (I see floating in the Internet) which is sqrt(#params * #active params) which would give sqrt(284B*13B) = 60B, so deepseek should perform better.
Seems like that equation really should weight total params and active params differently