That's mostly true(*), and we should work on making the opt-out a standard procedure where ever possible.
Instead, the public discourse centers around which minority can force the majority to follow their preferred terms in the social contract. At best, it's a zero sum game - most likely, it's a net loss for everyone if we include all externalities, especially the suppression of discussions about more individuality.
(*) DST is not a social contract, it's entirely made up by the law.
I'm thinking of a time I worked a university help desk and had to deal with a professor that considered himself a night owl and didn't understand why we weren't staffed during his preferred working hours of 10pm-8am. He was literally the only one on campus working those hours, besides things like security.
There's wanting to work a bit earlier or a bit later than everyone around you but there's a limit to how out-of-sync you can get.
You can take it in any direction you want, but I have pretty low expectations of schools willing to teach my kids from 10 to 16, or mechanics willing to fix my car at 20, or doctors willing to take my blood exam at 22 — all that just because that would be convenient for me.
The modern society started with the Industrial Revolution, when the people understood that there are some very tangible benefits when everyone starts working at the same hour, and started building around that. This building included discarding "local time" (an astronomical fact) in favour of standard time (social contract.)