I was suggesting something that could be done without changing the protocol itself.

From what I undestand, the terminal is essentially a concatenative text buffer extended with control commands allowing you to move the cursor (pen) to draw more sophisticated stuff, and update UIs.

The logical way to extend this would be to allow genericdrawing commands over the protocol, at which point the whole thing starts sounding like X11.

I'm sure the Unix people at MIT in the 80s saw this as the natural evolution from text terminals to graphics based ones.

Not saying this is a bad idea, but sounds suspiciously like a retread of history.

Or you might want to stop before that, as you suggested, at targeting fixed size cells for rendering, and keeping the console-like UX throughout. I'm sure there's a sensible intermediate step, but if you want to do things like displaying graphs or images (turning the terminal into something that's halfway between a terminal emulator and a Jupyter notebook), it'd be awkward to respect the character grid and the non-square characters themselves.

I'm curious what you think about how far this new protocol should go, and in what direction. You mention 'semantics', as describing behavior, not just what gets drawn to the screen, which neither X11 nor tty does.