alt Hacker NewsYeah, I'm seeing this at work too. It's capable of adjusting behavior to meet some narrow success criteria, but it can easily introduce bigger problems in the process in large coupled code bases, which it doesn't catch on its own. Automated tests help a bit, but I find I still have to steer it very strongly at work.
The trick is to take a step back every time you think you need to steer and figure out what tools the model had to have to figure this out themselves.
If there are couplings to other systems that are invisible to the model in certain situations, then you should make those visible to it!