> There is room for speedup where current implementations are slower than the proven lower bound, but not when they're already close to it.

Sure, but I'm obviously not limiting research to improvements on current approaches only.

We know the brain is far more energy efficient and sample efficient than current AI. There is clearly better algorithms out there.

The question is who will find those next big algorithmic improvements like the transformer architecture? Will it be AI or humans?

My bet would be AI.