You also need to involve better measures of agreement that are standard in the literature like krippendorfs alpha with ordinal metric. So many footguns in this methodology