During the height of the Triangle Trade, Europeans built a lot of forts to protect their interests along the Gold and Ivory Coasts; why these forts are considered castles by this site is beyond me (just for clarification: a castle is a fortified private residence).
Edit: I now realise that the site specifically says "Map of the world's great castles & fortresses", so I guess the site considers them fortresses, not castles. I retract my snark.
The title of the page is "Map of the world's great castles & fortresses". So those are latter.
There is no taxonomy of these things you could make that will not cause some eyebrow raising choices.
It counts Clarence House in London in its top 100 as a world famous ‘palace’ - it’s a Georgian townhouse.
Age of sail trading post forts seem a relatively uncontroversial inclusion in comparison.