>mapping to and from the token stream is harder, not easier, when the individual tokens carry less information.

this is exactly what I would expect. Also if you are training on code on the internet, what are the chances that you get these kinds of structural errors, especially on code in blogs etc.?

Lisp is known for being easy to drop a paren on accident so you saying not closing parens jibes with what I expect, that the LLM would predict wrong every now and then about if it should put one in a particular place.