My big issue with the M1 generation is the lack of more E-cores, which get assigned a lot of the OS background tasks. When working at the top of the RAM limit, I'd notice issues around lag to the point that I started looking at CPU stats and found those two measly e-cores completely maxed out for hours on end. Occasionally a crash, though that could be RAM related rather than CPU.

A bit of googling around seems to point that architecture as somewhat of a problem for others, and might be why Apple quickly moved to put more e-cores on the later generations.

Something to consider. Never managed to get proper distinct confirmation that this was the problem, but a (corporate) laptop upgrade did get me out of this pickle.