It's not particularly related. We have efficient quantum algorithms for RSA and discrete logarithms. Both are solved by viewing them as instances of the "hidden subgroup problem" over an abelian group.

Some well-known other problems are also HSP instances over non-abelian groups, for example

1. the learning with errors assumption (the main PQ thing people like) is a HSP instance over the dihedral group, and

2. graph-isomorphism is a HSP instance over the symmetric group.

LWE appears to be quite hard classically (SOTA attacks are 2^{~0.3n} time and exponential memory). Graph isomorphism is a famously easy problem outside of P, namely it is in quasi-polynomial time. So the fact that both are not in BQP doesn't say much about their relative classical difficulty.