Being able to restrict yourself is exactly what the right to enter contracts is.
Just like being able to commit yourself to showing up to work everyday restricts your freedom. But it's essential to getting the other party to give up something in consideration.
Of course, you can argue that consumers should be treated like legal babies that can't enter into agreements.
There's a difference between restricting your freedom and restricting your rights. When you enter an employment contract, you agree to give up some freedom in return for a salary etc. but it doesn't take away any of your rights. You still have the right to life, liberty etc. You still have the right to not turn up to work, though too much of that will likely cause them to terminate your employment. An employment contract that removes rights is typically not legal in sane countries (e.g. agreeing to become a slave).