I don't have the same experience, and I drink one cup of coffee (270 ml) almost every day. No agitation, no impulsiveness. I can drink coffee in late evening (let's say 8 pm) and sleep well. I guess I'm trying to say that we should not project our own experience on others, everyone is different.
In my experience, this is common among people with ADHD (myself, friends with ADHD, family with ADHD, psychologist’s patients anecdotal evidence). YMMV