TLDR: Like an LLM, the input is a string, but the output is not a completion. The output is a ranking of elements from a certain enum (e.g, [Yes/No], [A/B/C/D]). They use a technique called Reinforcement Learning for Calibrated Decisions (RLCD) instead of RLHF. Also, inference is a lot faster.

https://docs.typesafe.ai/primitives has a much better explanation