> removing all the fiber

Yes this.

Also reduced vegetable intake. In many cases the fiber from vegetables can offset some refined carbs intake.