People like S3 because they have hard engineering guarantees around bit rot and work well as a high level abstraction of a network filesystem with all of the low level failure recovery built-in. You don't have to worry about doing your own RAID configs. The bigger question is whether non-AWS services can offer the same level of guarantees. I have heard horror stories for example when it comes to downtime on Hetzner's S3 object store.
I am currently using Cloudflare R2 right now and if you see their forums, there's always the occasional post about objects going missing.
Meanwhile, just using disks and regular servers is still reliable and more so than ever, especially with some redundancy.
Consider your cloud costs.
I find that Ceph is pretty annoying to operate but if you do, it works fine. Keeps redundant copies of data on multiple disks, across multiple racks if your diversity is that wide. Supports erasure coding for effective redundancy less than 2x.
Don't go for the "object gateway" compatibility layer - just use raw Ceph if you're writing your own app.