I've worked as a rater before, working on doing the actual rating and also rating the raters. When you're a rater, you get very detailed instructions on what to do. When you're rating the raters, you're given the same instruction set so you have to rate them according to how well the raters followed instructions. I assume that someone else rates the raters who rates the raters, based on the same instruction following.

The job is easy and pays really well. If you want to keep the money flowing in, you simply follow directions. Standing out and being different gets you noticed, and not in a good way (because you're just a faceless rater id #667383737).