No one is claiming that all poly people have some ties to the AI community.

They make a claim about a specific community in San Francisco, that is tied together by a mix of AI doomerism and sexual relations. And it's not at all surprising or some farfetched claim that people who enjoy having sex with someone might adopt their ideas, at least publicly, for fear of losing that sexual connection.