In some cases, this was because of a single person's brave decision (Vasily Arkhipov prevented Soviet nuclear escalation in response to US aggression in the Cuban Missile Crisis, and Stanislav Petrov prevented it in 1983 when Soviet missile detectors misreported sunlight reflecting from clouds as 5 incoming American ICBMs -- credit to commenter folkrav).
In general, though, there's an incentive: Mutually Assured Destruction. But this is not at all some guaranteed, eternal thing -- it is absolutely dependent on both sides having time to detect incoming nuclear strikes and respond with the same before the first strike hits. When this fragile condition holds, and only then, both sides are incentivised not to initiate.
Unfortunately and fortunately, MAD and "launch it or lose it" are far-too-simplistic descriptions of the situations facing the decision makers. Unless a side's leaders are very narrow fanatics (vs. mere posturing as such for political benefit), "winning" an all-out nuclear war via first strike is a pretty shitty victory. Whether or not you believe in nuclear winters, the world would be a huge radioactive mess, with enormous social and economic disruptions, and your regime very widely blamed (and widely hated) for that. Ambitious underlings and rivals could see your removal from power as the obvious next step. Having to stay united against the (now destroyed) Great Enemy may have been a cornerstone of your regime's political stability.
Meanwhile, the leaders on the other side are aware both of those considerations, and of the history of near-disasters resulting from false alarms of enemy nuclear attacks. Making their own launch decisions much more complex.
They don't need to respond before getting hit unless you can hit their secret submarines too.
https://en.wikipedia.org/wiki/Letters_of_last_resort