You don't have to remove them - you can move things that are not important to every prompt to aseparate small docs that are referenced in agents.md with ,"when needed, reference these files:" and list them in form * relpath - content summary" or similar.
. When you want those instructions to be followed, mentiont them in your prompt. "Test this following procedures on docs/test.md". (I've found I don't have to do that extra instruction in al cases depending on model)