That's precisely what I want. I've got all the parts other than the SBC. Even designed my own keyboard with this in mind. You can do it with a pi zero 2 but going from mini-HDMI to full-HDMI through an adapter to USB-C to only then pop out too the XReals is a complete mess of cables and little boxes, not least of which is caused by the HDMI->USB-C adapter needing to be independently powered. I'm tempted to print up a case to hide the mess but it's still, you know, a mess.

Also the zero 2 is a little under-powered. It's fine as far as it goes, it just feels like there's probably performance on the table with a more modern chipset.