This makes total sense to me. Everything that makes up the state of your project should either be part of the versioned repo (git) or not be part of the project.

I created a little Github Issues replacement for myself that puts the issues within the repo so that the work and the todos stay in sync. https://github.com/steviee/git-issues

And I bet there's numerous other projects like that.

Hope you get your submarine, man! ;)