The term was adopted in Star Wars and has been used in science fiction before and after it too. It probably sits somewhere on some scale but it has always been used in literature as a term for machines and from that perspective I find it appropriate.

A machine does not have feelings. If you call your power drill a piece of shit you are not overstepping any bounds either.

Droid don't have race. Moreover, even if they did, that does not extend the slur to a particular group of humans who the word was not even being used towards.