The particularly worrying thing here is that they're now going to be gathering training data for a conversational model on _how to influence people effectively even when they already know they are being influenced_. Even more than RLFH already does. "We had to build the Torment Nexus so our children could eat" is not a good reason to build the Torment Nexus. The fact that they are not committing to not doing this tells me that either no one thought about what else this could be used for, or the short term gains are all they are thinking about.