Story points are unicorn dust that crumbles under any attempt of serious optimization. The fundamental problem is that SP is not an objectively defined metric. If we come under serious pressure to improve velocity measured by SP, there's nothing to stop that initiative from trickling down into the SP estimation/measurement. SP works fine as long as you don't look too closely at it.