Clock speeds used to be going up in a straight line (the normal "interpretation" of Moore's law) - but once the P4 hit a (kind of useless 3.8GHz) we leveled off for decades.
(To make it clear, straight line on a log scale. Exponential on a linear scale.)
More specifically, it was the end of Dennard scaling [0] that killed off the growth in clock speeds in the mid-2000s.
[0] https://en.wikipedia.org/wiki/Dennard_scaling