logoalt Hacker News

the_harpia_ioyesterday at 8:00 PM0 repliesview on HN

agree tests help but they only catch what you test for - and honestly a lot of codebases have patchy coverage at best. the bigger issue is when the AI misunderstands the task itself, like implementing the wrong thing correctly. tests won't catch that if they're based on the same misunderstanding. the reward hacking point is real though, seen that where it just makes tests pass by changing the test