Unfortunately, I don’t this will work until we have robot secretaries that can automate updating paper wall calendars and documents and books scattered around a room.

The only compromise would be a limited area like a physical desktop that had affordances like an overhead camera and some form of paper output.