This is how I see/use brew as well, and being able to just blow the directory away anytime and start over if need be is nice.

It's not a "system" package manager, nor was it ever meant to be. Its supplemental. I've also found it valuable on the various immutable linux distros.