> Also, while it's easy to calculate word frequencies for stuff like newspaper articles, there aren't any good statistics (last I checked) around just normal everyday conversation. Because that stuff isn't getting recorded and transcribed.

Linguists have resorted to spying on people in parks and shops and restaurants, to get "natural" language unperturbed by observer effects, audience tailoring, the filter of writing, etc. (In most cases that would not be considered ethical today!)

The gold standard is something like: get acculturated to the language community you wish to study, make some acquaintances and friends, somehow finagle an invite to their homes or workplaces or social venues, and then with their permission, record their conversations as unobtrusively as you can.

There is so little data on how language is really used it is quite bedevilling. As soon as people know you are listening they change how they speak.

Oh, this is such a fun rabbit hole.

There are plenty of programmers who can write a commit message in almost perfect English, yet don't know the difference between lettuce and radish, and can't tell what a little sheep is called.

My favorite area to catch people on is religion. Many C1 English speakers who grew up religious, some of whom are still devout and go to Church every Sunday, would be unable to hold even a basic conversation about their religion in English.

The unfortunate side of this is how non-native speakers of English swear much more often. I got better at it, but I suspect I'd still struggle in American corporate culture.

> If it's for reading the newspaper, you don't ever need to know "bathroom", but you sure do need to know words like "congressman".

Unless you're reading a newspaper from the last 5 years or so - in which case, the word "bathroom" becomes surprisingly more relevant.

There is also a whole different language used to talk to toddlers and small children that native speakers know but very few adult learners ever pick up. And slang used among teenagers and young adults that often times not even their parents understand fully.

Beware though that only learning words may leave you confused as in my experience Americans tend to overload the same word with a different meaning depending on the context. They also often resort to idioms and cultural references.

For reasons unknown to me, instead of using the word with the meaning they probably are trying to convey, they rather use vague analogies or metaphors.

It always made me question if Americans in contrast to the British are a bit afraid to be precise.

I think a lot about this. It's funny how there are certain domains of language, familiar to all native speakers, but that you are simply not likely to learn as an adult language learner, at least not without diligent and focused study.

I live and work in a foreign country, and have a modest but functional grasp of the language here. Which is to say, I know how to say 'sustainability', 'union-negotiated collective agreement', and 'offensive conduct in the workplace'.... but if you asked me the words for 'cedar', 'robin', 'pond', or 'linen' I would be struck dumb. And yet presumably every ten year old I walk past in the street would know those words as comfortably as I knew them in English as a ten year old in England.

After years of reading technical and scientific materials, listening to lectures and youtube tutorials and learning English exclusively through them I hit a wall when I watched a cooking youtube video for the first time.

I was pretty confident in my English but I couldn't understand half of the sentences since I didn't know any of the vocabulary related to cooking. From tools (spatula) to ingredients (parsley) or even actions (frying). My delusions of proficiency shattered as I found out my everyday vocabulary is smaller than an elementary schooler.

I read years ago that:

1. TV vocabulary is 2,000 words

2. High school vocabulary is 10,000 words

3. College vocabulary is 30,000 words

4. English language has 1,000,000 words

This is good news - you can be proficient in a foreign language by learning only 2,000 words!

I tried to teach 'magicE vocab', sorting them by difficulty level for an English language plan, and got them easily arranged. For example, sham/shame and slide/slid are for hard to learn level, while ate, pale, kite, are for the easy level.

Kinda like how there's no average-sized airplane pilot

> While if you are living somewhere, it's really important to know a lot of basic supermarket items that you wouldn't prioritize for other usages.

Why? Having spent a good amount of time living in Shanghai, I found it important to be able to understand menus. But there's no pressure to know the words for supermarket items; you can just go to the supermarket and look for the item.

Otherwise your point is correct; all semantic words are equally difficult and which ones you know depends on the things you like to talk about. Grammatical words are more difficult, and more important, but this is so widely understood that language-learning material already treats them as an entirely separate class of things to learn.

> Also, while it's easy to calculate word frequencies for stuff like newspaper articles, there aren't any good statistics (last I checked) around just normal everyday conversation. Because that stuff isn't getting recorded and transcribed.

(1) You seem to want COCA, which includes a bunch of transcribed telephone calls.

(2) Word frequencies are still the wrong concept. If you want to understand a particular document, you need to understand almost all of the words that appear in that document. (You'll be able to learn some of them from their use in the document.) If you decide to learn a list of "frequent" words, you're unlikely to be able to understand more than a couple of isolated sentences in any given document.