> I always wish I could replace the UI with my own embedded editor (emacs in my case)

Consider his his older project instead? https://overtone.github.io/

He even used to perform using emacs before he created Sonic Pi.