That sounds cool. For my uni project 10 years ago I chose to use neural networks to detect guitar notes, but not by pitch, but to detect exactly which string/fret was played (because you can play same note on different strings). It worked okish with a very basic network, quite suprised. I made a small game you could play with your guitar.

Since then I wanted to develop further such a game, and now I'm almost ready to release https://ultimidi.com , check it out, it's not really a rhythm game, but my goal is to add more mini-games to it (including rhythm games) and more instruments (now has piano, but planning to add guitar, ukulele too, not sure if drums would work too).