Cheap, fast and somewhat capable models are insanely effective at highly verifiable tasks, even if their overall capabilities are under SotA. You can leave them banging their tokens against a wall, and come back once their attempts are verified. And youc an always clean up afterwards, once a task is solved, if needed.
I've had a lot of success with dsv4-flash on these type of tasks, where it's easy to set a threshold for the task, and just loop it until that threshold is reached.
oAI's Luna play is really good. They've slashed the prices, the model is somewhat capable, and you can use it both for these kinds of long horizon tasks, or you can hand-hold a bit and get extremely cheap results out of it. And they get to keep devs in their own ecosystem.
alt Hacker News
Cheap, fast and somewhat capable models are insanely effective at highly verifiable tasks, even if their overall capabilities are under SotA. You can leave them banging their tokens against a wall, and come back once their attempts are verified. And youc an always clean up afterwards, once a task is solved, if needed.
I've had a lot of success with dsv4-flash on these type of tasks, where it's easy to set a threshold for the task, and just loop it until that threshold is reached.
oAI's Luna play is really good. They've slashed the prices, the model is somewhat capable, and you can use it both for these kinds of long horizon tasks, or you can hand-hold a bit and get extremely cheap results out of it. And they get to keep devs in their own ecosystem.