alt Hacker Newspostfix, dovecot, rspamd on debian stable, with lots of custom configuration. The main problem is not really the server side or getting it set up right, if you know what you're doing, but outbound delivery if you're sending SMTP traffic from any kind of bulk hosting/VPS/VM/colo provider. No matter how impeccable your spf, dkim, dmarc is.
I run the same, except on FreeBSD. I have my own ASN and an IPv4 /24 block with a good reputation dating back the 90's. Basically, it looks like a corporate network, not some random VPS, even though it's run out of a basement colo in the middle of suburbia.