Yup, that's exactly it:
- get the gravitational constant with these two known masses
- then can deduct the mass of the unknown Earth by its interaction with other masses (say the "g" gravitational acceleration value)
- then from the mass and the otherwise measured size of Earth the density pops out
More details in good ol' Wikipedia: https://en.wikipedia.org/wiki/Cavendish_experiment#Derivatio...
It's also a notoriously difficult experiment to perform. When I did it at university, the value of G I got was out by an order of magnitude - and that was considered a good result!