We don't use most of them because no one could learn them all. There's more than 100,000 hanzi recorded in Chinese. Some quick googling suggests educated Chinese people know 8,000, and the average Chinese person knows around 3,500.

So 3–8%.

Nothing to do with pretention and everything to do with impossibility :D