The real question is, do current training methods allow one to learn this when training on enough, say, assembly, or can the weights only be achieved via hard-coding?

A fun and important thought experiment about the the space of nerual networks vs the image of current training methods.

Most discourse conflates neural networks and trainable neural networks.