This is true for the most part, unless you adjust keyboard settings in System Settings to let all UI elements be focusable with the keyboard (for tabbing between UI elements). I think it used to be under "keyboard" but they might have moved it in the recent Control Center reshuffle.