Haven't you? There's quite a bit of closed-source software based on permissively-licensed code. You can get the upstream source for the permissively-licensed code, but not the modifications made to it. Windows (used to) use some BSD-licensed code for a few network utilities like nslookup.
I guess I considered that completely separate from open source. For instance, I've heard that MacOS is "based on BSD," but since MacOS itself is obviously not open source, it's not an example of open source software that doesn't provide the Four Freedoms.