alt Hacker NewsThe biggest discovery might actually be that they ignored regular caching rules and kept invalid cache suffixes and it didn't hurt performance
The biggest discovery might actually be that they ignored regular caching rules and kept invalid cache suffixes and it didn't hurt performance
I wonder how bad the performance would be if they plain ignored the whole rotary encoding dance and just back-filled precisely the parts of the cache that changed directly. Would it break the model? Confuse the model? Or would the model internally correct for it?