alt Hacker NewsI thought it pretty clear that the code was generated by the same agent that received the testing prompt, so there were no constraints on the code structure, and the testing strategy was known at the time the structure was generated.
I thought it pretty clear that the code was generated by the same agent that received the testing prompt, so there were no constraints on the code structure, and the testing strategy was known at the time the structure was generated.
But what's the initial setup? Was it greenfields every time? And _how_ were the different testing frameworks used? What's the AGENTS.md there? So many things can influence these tests in positive/negative ways that are not included in the write up or results. Seems like a lot of effort without much in the way of actual helpful detail.