Yes it just collapses eventually — never stabilizes. The training process is flawed, I suspect it has to do with the fact that some weights blow up over time, you can see in “weights” tab.

But at around 4K avg score you should see it solve the env almost every time.

Just a demo :) optimized for speed over stability.

Reward structure: Step: -1 Dot: +100 Win: +1000 so ~4k is max theoretical score on 6x6.

I think I noticed it reach “end game.” The snake reaches a point where, if it gets any longer, it is out of squares and hits its own tail. So it finds the route through the squares that it can infinitely loop, never eats the ball, and score starts dropping and goes negative.