logoalt Hacker News

sigmoid10yesterday at 9:59 PM1 replyview on HN

It's fast if you want to automate things that run independently or overnight. It's slow if you want to iterate code together with it.


Replies

Wowfunhappytoday at 12:10 AM

If you were iterating on code with a human partner, how fast would they write?

Like, obviously faster is better and the entire point of computers is to do things faster, but I find it kind of surprising how many people consider 9 tokens per second—heck, even much less than that—to be unusably slow. It's still automating a class of task that virtually no one before 2022 was able to automate, and it's faster than basically any human can write code.

I guess the better question is, why would you use a 9 token per second system when you could use a much faster cloud model. Obviously, if you want speed, that's the way to go. But a lot of people seem to find the idea of sending their requests to a third party server untenable. If 9 tokens per second is the best you can do—I don't know, that seems usable and useful to me.