alt Hacker NewsI suspect that Dots and Duet are very high margin drivers for the labs. Personal assistant chat sessions don’t have to be realtime, so workloads can be scheduled on low priority batches and be waited if higher value work comes into a queue.
Inference providers are always trying to fill inference batches and having a stream of low priority requests can help pack batches that might have gone empty, without raising the latency of higher priority requests such as live chat sessions.
While Dot and Duet certainly need to be useful to work out as products, I think the gross margin mechanics are just as interesting.
Replies
I just watched a YouTube video from How I AI[1] doing a bit of a preview on Duet. Their point was quite the opposite. The Duet agent is so slow that it's not actually particularly pleasant to use, even if it is smart. I tend to agree, and think speed is a key feature here. If it's not quick, it risks feeling disconnected.
I have, against my better wishes, rather given in to Grokbot over the last couple of weeks. It does feel very nice to use for assistant type tasks. At least part of that is because it feels pretty responsive.
And there's also a lock in mechanism. With regular chat, you can simply switch to another provider, and with a harness or API, switching is pretty easy too but here, it's more difficult.
It might become a triple-whammy benefit for them. More efficient compute, as you said, but also a bot capability creates a better moat around their infra ecosystem and the amount of training data that can come from a bot ecosystem is significant.
Models are becoming increasingly interchangeable but setting up and maintaining (!!) the ecosystem still takes expertise. Not everyone wants to host their own MicroVM infra and would happily pay Anthropic/Grok/OpenAI to do it for them.