Suppose someone could break RSA / Elliptic Curve Crypto, mathematically (because the hardness assumptions were flawed).

This person doesn't need to store the private keys, and has the luxury to recompute them on the fly when needed!