If companies are taking raw materials worth more than zero, and turning them into clothing worth less than zero, then I think deterring them from doing that is beneficial to society overall.

The worth is zero because the producer doesn't pay for the externalities (pollution, landfill usage etc). So essentially it is "free" because it is subsidized by everyone.

The "headache" is just : produce what you sell, sell what you produce, don't fill the world with your shit.

Or rather, since we know fast fashion is horrible because of the things you just said - it forces a more thoughtful approach to production.

Also: this will lead to it being harder to find clothing in your size in the EU (since each size is a new sku and must be inventory managed per the law)

If the headache causes companies to improve their product pipelines so that there is less waste then surely there will be less recycling.

Discouraging superfluous production is not nothing.