I get what you're saying, yeah. I would predict (not an economist, some guy on the internet) that unless you outlaw working over a certain number of hours, people will work more hours than ideal and it would push everyone else to do the same. In other words, having lots of non-working hours is an unstable position, as people work more to satiate their unlimited wants.

You could outlaw or heavily disincentivize working over a certain number of hours (overtime is a step in this direction), although my concern would be that artificially limiting productivity like that would be detrimental. We still need people doing productive things, so slashing their hours might be a Chinese "backyard furnaces" sort of situation. That said, some people think half of our jobs are bullshit anyway.

To your credit though, we shouldn't let perfect be the enemy of good. Maybe 36 hours is better than 40, and so on.