I don't think fair comparisons are really possible across different processes. There's a lot of interaction between silicon process, architecture, and workload.
The fairest comparison is between chips from major vendors in the same year. If you're just doing FP32 matrix multiplies, a 2026 nVidia GPU is ~1 pJ / FLOP, an Apple M4 is ~30 pJ, and a 2026 Intel Xeon is ~50 pJ. The quality of process and optimization is a little different between those 3 companies, but not by as much as 2x. But from those numbers, you can reliably conclude that the GPU architecture is much better for matrix multiplies.
Yes, that is surely true. What about small in-order scalar cores like the PULPino? Maybe the M4 and Xeon are spending a lot of those picojoules on OoO, caches, speculative execution, etc.