Or he believes that other labs might get there first, and he is working to counter that threat.
This is the Manhattan Project again.
Yeah, super ethical lab earnestly believing AI has 10% chance to kill all humans within a decade and doing utmost to protect humanity is partnering with MIC and Palantir in particular. Sounds about right.
Before you tell me about how the CEO has taken a principled stand: on record, he had no problem using it against 95%+ of humanity outside the U.S., and was only against fully automatic AI killing machines, and only citing the technical reality of then-current gen tech, so one should read that as human-rubber-stamped AI killing machines are totally fine with him.
Or he is paid >1M USD per year
How exactly does that work? The nuclear system of MAD relies on physical threat, lab A achieving ASI (artificial scary intelligence) does not prevent lab B achieving it.
I would like everyone involved to be a lot clearer about their threat models, with plausible series of clearly linked steps, rather than just sounding like a Vernor Vinge novel.