We built something quite similar 6 months ago called PEAK (Proof of Effort And Knowledge). Blog post and YT video here: https://substack.com/@jeffksmithjr/p-195763168

Our version has its own TUI and was intended for semi-adversarial scenarios like OSS contribution and enterprise. Thus a lot of anti-cheat tech under the covers (including invalid questions).

My guess is that a lot of folks have built something in this vein. It really made sense earlier in the coding agents cycle. These days, I dunno. Seems like we need to more fundamentally rethink the relationship between coding agents and human understanding of the implementation.