It's not that iphone keeps that 4; it's that iphone pretty quickly starts accepting the next round of input after 1123 and then you type 4.

Perhaps a longer pause will prevent you from typing that 4 but that also means other people who mistype their passcode have to wait longer to retry. It's a tradeoff. I suspect there are more people who type the wrong passcode of the correct length than the incorrect length.