It's not like we are born with a "phone number library" skill. It's a specific skill that's developed to meet a need. It's carved out of the grey matter, a little table for storing 7- or 9-digit numbers. But the grey matter is still available for other uses.