Step 2: Prompt the agent to comment out the "when" code and check if the tests fail. (If tests don't fail it's a poor test)
Step 3: Prompt the agent to write mutation tests to harden the suite
Step 4: Prompt the agent to write property-based tests.
Step 5: Prompt the agent for 100% code coverage
Step 6: Keep a look out for small issues with steps 2-5 above and refine a Skill or Rule to package the workflow and plug gaps in the model's current shortcomings.
Step 7: Dedicate some time to manually reviewing and thinking about interesting edge cases worth testing.
---
If you follow these steps, you will have better test coverage than 99.9% of any code an engineer ever wrote pre-AI... and it will be almost completely automated.
My team's incidents and bugs are DOWN this year while we're shipping more.
Leveraging AI effectively requires a new paradigm of interaction. The name of the game is to use techniques and strategies and guardrails that create an environment that produce good software.
It’s also a great way for AI to keep creating backwards compatible bloat code when refactoring and whatnot because “tests are failing” when those tests are testing absolutely silly stuff (LLMs have a tendency to want to validate exact output rather than behavior).
I don’t think anyone has figured out the proper solution yet, I personally try to get the LLM to write code similar as I would have written it myself.