Jellyfin after the 10.X update changed their database. This introduced a major performance issue with SOME large music libraries. There doesn't seem to be a hard set of criteria besides "some large music libraries suffer from bad performance degradation". I finally managed to upgrade after just brute forcing it and trying several times. It finally just worked.