alt Hacker NewsI think you're overlooking an important point here. While the token consumption might feel extremely high, I think it's a bit deceiving.
Due to this strict harness, we were able to use extremely dumb models efficiently. I think at least 200-300B of tokens (maybe even more) were processed by Luna.
Given that Astra for example is 100x more expensive per token than Luna, a high token consumption doesn't necessarily mean high cost. So these 300B Luna tokens would equate to 3B Astra tokens and roughly 15B Sol tokens.
Unfortunately a huge part of the logs is lost, so I can't do a detailed analysis on which models were used, how much they cost, etc.
Replies
I don’t recommend always using Astra for RE but in my experience I’d use Sol as minimum. As of recently, perhaps upgraded Terra is good value, but in general it seems like you want the smartest model that is economical to run 24/7 and Astra isn’t economical.
> Due to this strict harness, we were able to use extremely dumb models efficiently.
With all due respect I seriously doubt that. What are you doing that's different? Almost every test between just Opus and Opus conducting cheaper agent swarms ends up with Opus getting better results at a lower cost. People set up harnesses just like yours and see this.