You might want to read my comment again. :) If you use labels, the app will have full access to the associated contacts, not just to their names & phone numbers.
I'm annoyed at everyone who shares my name, phone number and any other details with Meta. I never consented to it. The behavior of their app slurping up your contacts database is despicable.
This doesn't answer your question, but in case it helps for others out there: it's possible to use WhatsApp with no access whatsoever to your contacts and I used it that way for years. Connecting with people is slightly jankier but it still works.
So it's not about labels, but you want the ability to restrict the fields an app has access to rather than an all or nothing – full access to a contact or none at all?