Yeah, that's why my idea was to use neural nets trained on manually labeled data, instead of relying solely on pitch detection.
Also, probably AIs nowadays can help a lot with generating training data, and even improving detection.
That being said, even SOTA voice recognition has only like 95% accuracy, so it gets wrong 1 in 20 words. Imagine if transcribing a melody and 1 in 20 notes are wrong, for a fast melody that would mean a wrong note every few bars.