A few things:

First, every BloomyBot response streams through an inline safety classification, and then an independent second-pass audit sweeps every stored message — a deterministic rule layer merged with a separate LLM classifier, taking the highest-severity result. It flags three categories (crisis, distress, inappropriate) across every language we support, and confirmed flags alert the teacher and school. We also constrain what the model is allowed to do: the tutor won't reveal answers, and it's completely absent from Summit assessments — so an LLM output never decides whether a student has mastered a skill. That comes from scored assessment performance against our 90% threshold.

Second, a nightly cron job runs across all BloomyBot interactions to flag erroneous outputs (so far, none) and safety alerts (these happen real-time anyway, but the cron job is a backup).

Third, I sample the interactions manually. I've read most interactions so far (although this will become much harder with more scale).

Finally, we are building structured evals to draw inferences across grade levels, skills, common misconceptions, etc. Ideally we will be able to identify the optimal level of scaffolding to help students achieve mastery, but that will take more data and time.