Do we actually know enough about neural networks to be able to say how many parameters you need for a given performance? I’m under the impression that we’re still in the phase of trying things out and we can’t rule out different architectures massively reducing computational needs.