The problem is that unlike data prefetch, the so-called "instruction prefetch" is not actually prefetch at all.

It's simply speculative execution. Which doesn't look any different to regular execution. The fetcher has no idea that its predicted branch is about to invalidated and flushed, otherwise it would never have issued that fetch.

Actually, on a modern OoO core, [0] it's very rare for the instruction fetcher to not be doing speculative fetches. Even when it's not predicting a branch, the fact that it has "predicted" the lack of a branch is speculative in itself. It assumes it didn't fetch a branch in the last cycle, but it can't be sure until after instruction decoding, which takes at least 2 cycles (more on larger L1i caches).

About the only time the instruction fetcher is not doing speculative fetching is for a single cycle after each miss-predicted branch.

[0] Or even something technically in-order, like the Cortex A53 cores here. They might issue in-order, but because of how they implement dual issue, they look somewhat close to a simple OoO core... I suspect they actually do register renaming. And (most importantly) importantly they have a branch predictor.