I was taught the same thing. In particular I kept being told that gesturing is an indication of a poor vocabulary. So I taught myself to never do it.

I do not mind gesturing itself done by other people, but I’ve found that it correlates a lot with violating my personal space. A lot of gesturing ends up being done right in front of my face, or just too close to my body in general. And that annoys me and makes me lose respect for the person doing that.