> I just wish there was some kind of ACID test suite for XMPP or something else to reproducibly validate spec compliance. Maybe a test server or similar as a reference implementation. This way client or server maintainers would have to run their programs against the official test server to increase their compliance stats.

This is exactly what the Compliance Suits are for, and the XMPP Software Fundation is taking care of telling all the clients what they misses directly on the official website, for example: https://xmpp.org/software/movim/