You could use PEFT? Operating on only a subset of weights is fairly standard practice nowadays …
Yes I used LoRA and it’s fine but I’m not convinced the model doesn’t end up more stupid and less general
alt Hacker News
Yes I used LoRA and it’s fine but I’m not convinced the model doesn’t end up more stupid and less general