> hey currently monitor it to prevent users from using their models in certain ways they consider unsafe or bad for their business

Can't they just monitor the text outputs as well?