alt Hacker NewsI would have loved to see more of the harness engineering shared as code. Instead we are left with only the outcome.
I guess that makes sense since the harness is the product in the case of Cursor.
Replies
I call this junk "meta-agentic engineering", it reminds me of people who have the coolest nvim configs, spend hundreds of hours customizing it but ultimately get less work done than the guy with minimal workflows, if any at all.
I look on twitter and it's just people building tools for agents to use agents, some weird customization loop going on in the LLM space right now. Ultimately these are trends pushed on us from model providers because they 10x token consumption. Its literally just BS trends to increase revenue at these companies, most of it is largely useless.
I don't think there is a lot test harness, given there is a golden implementation (sqlite3), with clear interface (SQL, and sqlite3 db files). Once you slop crappy implementation, then you can create one group of agents finding gap where slop doing wrong vs golden, and another group of agents filling the gap meeting the conformance.
I assume they did similar, as long as test case(or spec) can be parallelized. Having parallel execution / testing environment is crucial if we need 1000 commits/second throughput.
I would not surprised they have 1+ millions of specs.
I would bet they won't release generated specs - actually this will be a valuable asset like https://sqlite.org/th3.html