Compression doesn't really work for model weights.

Model quantization and model distillation are two techniques to reduce model size.