It could probably go quite a bit faster using Arm SME. The entire network could fit into the ZA register.