> but it seems less likely to me than before that the types of math/science discoveries will explicitly unlock better software performance.

You seem to overlook a simpler barrier. To make these advances, they have to be possible. A 15% improvement in GPU kernels doesn't evidence that significantly more improvement has been left on the table.