Elseh I'll grant, but it feels a lot less natively English, and you can't count on an American clerk or census taker to care about accurate transcription. It may have been closer to "that's not a name in English. Pick either Elsie or Elsa".

And yeah, Elsa was common too.