Already exists, it's what people refer to as "male olympics". As far as I know, females aren't banned from competing. It is just that they don't stand a chance in most disciplines. The whole point of female olympics is to keep males out.

Why not ignore gender labels and go by chromosomal configuration? There could be XY and XX [1] olympics. And then there should be X, XYY, XXX, XXXY, XXYY, and all the other possibilities [2].

There is more complexity than the binary in the expression of sex in humans.

[1] https://en.wikipedia.org/wiki/Sexual_differentiation_in_huma...

[2] https://en.wikipedia.org/wiki/Intersex