I have the opposite experience. Phone calls almost always leave me feeling that something important has been left out. But there is no record so I can't re-read it to find out if I forgot or if it was just not mentioned. Luckily I rarely had to deal with such things because most of my work required written specifications and technical standards anyway, plus our teams were scattered in widely different time zones so the windows for live contacts were small.
It's very easy today to transcribe and summarize every call that you have.