logoalt Hacker News

djx22today at 12:19 AM1 replyview on HN

Not sure what everyone's experience is but I find 5.6 Sol to be a great liar. Reported success on a half done job and left things in a broken state after having quite a few back & forth followups on the initial prompt to clarify the plan. Didn't experience this with 5.5. Opus 4.7 and below sometimes did it but they fixed it in Opus 4.8. So, overall, the initial experience has made me think that this model will be a lot more stressful to work with just because the level of trust that it actually completes the task is now much much lower.


Replies

mnickytoday at 12:30 AM

May be related to this from METR evaluation:

> GPT-5.6 Sol’s detected cheating rate was higher than any public model we have evaluated

show 1 reply