> It seems obvious to me that AGI will certainly lead to ASI. Unless human-level intelligence is something like the speed of light?
It may just be that human-level intelligence is basically "enough" for performing tasks in the natural world, and having more of it doesn't really help except for synthetic, intelligence-shaped tasks like coding. Our perception that intelligence is some kind of unbounded problem-solving system that can do anything if you have enough of it is kinda suspect. There's a good likelihood that AI will wreck the (also intelligence-shaped) social fabric but will be of surprisingly little consequence in the realm of physics and energy.
I think the idea that an intelligence can be "general" is just a conceit borne from the perception of our own success, but mathematically that doesn't actually make any sense. No optimization strategy works on every problem. There must be things that can be intelligently designed that cannot evolve, and things that can evolve but cannot be intelligently designed.