Tools like Playwright and Maestro can already give you a small taste of what that would look like.

But overall I agree, LLMs are currently awful at being beta testers. They miss the most basic stuff that any human would immediately catch as being poor UX, and for all their visual prowess they are terrible at auditing UI.