logoalt Hacker News

c7byesterday at 10:04 PM0 repliesview on HN

Regarding the first, parallel requests to the same loaded model seem to work pretty well, I'm trying to find time to look more into it myself, but this may be something that might already be within reach for local models.