alt Hacker NewsFor me creating the exact sound is not very interesting from sound designing perspective. You can always sample the real instrument.
Like physical modeling synthesis, the interesting part is to compress the sound to some parameters that you can tweak and generate new sounds
Another approach is VAE, which also you give your some latent embedding, you can tweak the embedding to generate new sound. However the meaning of this embedding is not explicit.
"You can always sample the real instrument."
This doesn't really work on instruments like guitars. Open D sounds way different than fretted D on the E string. Timbre changes with position and it's one of the ways I determine where a player's hands are on the neck when I'm trying to play their song.