This would've been true a while back prior to major browsers checking for certificate transparency log entries as part of the handshake.

So, I do agree this could be problematic, on specific cases where people not using a major browser were accessing resources some is interested on, on a network they have control of. Makes the surface quite small.

The threat here is more so for 'backend' services than regular browser users.