Hi, thanks for sharing.
Have you considered using usb-hdmi capture dongle instead of using hdmi-csi. That is my idea, you bring your own capture dongle, plug it in and good to go. The esp32-s3 (and P4 i guess) support usb host, and there is uvc example from esp-idf.