You might have an older GPU that doesn't work with wayland like me. My Radeon HD5870 also won't do Vulkan and anything wayland has never worked properly for me.
I have the same issue using either an older AMD card and an RTX 3 series card. Both are fine with X11.
I'm not questioning what you are saying, but Wayland's only requirement is DRM, which is a Linux kernel capability that is supported for basically everything that you can push some sort of display cable into. It's for buffer management, and X was also ported to use this API.
Unless you have proprietary X server blobs, you have mostly the same low level route in either case to display stuff, so it's on the exact compositor you have tried, not on the wayland protocol.