My general take on any AI/ML in medicine is that without a proper clinical validation, they are not worth to try. Also, AI Snake Oil is worth reading.
Clinical validation, proper calibration, ethnic and community and population variants, questioning technique and more ...
Exactly. There's a lot of potential, but it needs to be done right, otherwise it is worse than useless.