logoalt Hacker News

yuanBuildsyesterday at 11:53 PM1 replyview on HN

You definitely pointed out the tendency of those frontier models to utilize external tools, in your case, the Stockfish chess engine, to solve some problem that the model itself is not good at. However, I am not sure is this a "hack". Your prompt does not explicitly prohibit the model from doing so, and, using the tool (some chess engine) is also a capability of the model.

Actually, the ability to use those external tools is one of the reasons of the excellent performance of the frontier models.


Replies

chr15mtoday at 12:00 AM

Yes but most people would consider this cheating. If you ask an LLM to fix the tests, you do not want it to change the failing tests to display little green ticks.

show 1 reply