logoalt Hacker News

zambellitoday at 2:12 AM0 repliesview on HN

Do you mean catching errors as tokens stream back versus waiting for the full message? If so, then no I hadn't looked into that. This was mostly geared towards local models so token cost isn't really a big deal, though latency might be.

And if you didn't mean that then please elaborate :)