alt Hacker NewsIs integrity checking of weights even common right now? What about upstream integrity from where we're getting weights? The open-weight supply chain is ... wide open.
Makes me think about hostile hot-patching of weights of models we use, maybe they exfil data, perform tool use, or have triggers for specific types of responses. Would we even recognize what's going on?
We've known for some time (3+ years) that models can be trained with backdoor behaviors and hidden triggers, this lowers the difficulty.
Of course, "academically known" isn't quite the same as "executives willing to admit it in the face of IPOs and quarterly-earnings reports."