I have the same issue using either an older AMD card and an RTX 3 series card. Both are fine with X11.