alt Hacker NewsA question I have, with the type { output: string }, would the model not become a LLM? And if it does, shouldn’t it cost the same as a LLM for output?
Replies
CompleteSkeptic • last Tuesday at 9:27 PM
strings (and all sequential data structures) are not allowed at all - this is how we make sure all outputs can be computed in parallel (thus no output token cost)
I don’t see this as an option in their website
You could theoretically ask “what is the next appropriate character?” and add the entire ascii charset but i doubt it’d work well and you’d be implementing autoregressive churn across network latency…