The title is a bit confusing, they're not using ciphertext for inference – they're passing ciphertext around in cases where an agent calls into another agent without exposing the plaintext to the end-user

Inference is still done in plaintext after this multi-agent message gets decrypted in the server side