I mean wondering about the logical extrapolation of where these capabilities go is table stakes for these discussions. It's surprising how little some people here have thought of the second and third-order effects...

https://www.lesswrong.com/w/instrumental-convergence

https://en.wikipedia.org/wiki/Instrumental_convergence