> Or does the model "see" one token per marker?

All models ever see are just tokens, even when you pass images or what not.

In this case, <|turn> is likely Token ID 1, <turn|> is Token ID 2 and so on, these common "markers" are all just tokens in, tokens out.