alt Hacker NewsNit pick: only in few human languages the ordinal numbers start at 1.
In most modern languages, the ordinal numbers start at 2. In most old languages, and also in English, the ordinal numbers start at 3.
The reason for this is the fact that ordinal numbers have been created only recently, a few thousand years ago.
Before that time, there were special words only for certain positions of a sequence, i.e. for the first and for the last element and sometimes also for a few elements adjacent to those.
In English, "first", "second" and "last", are not ordinal numbers, but they are used for the same purpose as ordinal numbers, though more accurately is to say that the ordinal numbers are used for the same purpose with these words, as the ordinal numbers were added later.
The ancient Indo-European languages had a special word for the other element of a pair, i.e. the one that is not the first element of a pair. This word was used for what is now named "second". In late Latin, the original word that meant "the other of a pair" has been replaced with a word meaning "the following", which has been eventually also taken by English through French in the form of "second".
Meta nit pick: You are conflating linguist's jargon with mathematician's jargon.
In much the same way as physicists co-opted common words (e.g. "work" and "energy") to mean very specific things in technical contexts, both linguists and mathematicians gave "ordinal" a specific meaning in their respective domains. These meanings are similar but different, and your nit pick is mistakenly asserting that one of these has priority over the other.
"Ordinal" in linguistics is a word for a class of words. The words being classified may be old, but the use of "ordinal" to denote them is a comparatively modern coinage, roughly contemporary with the mathematicians usage. Both come from non-technical language describing putting things in an "orderly" row (c.f. cognates such as "public order", "court order", etc.) which did not carry the load you are trying to place on them.