My understanding (which could easily be wrong), is the big difference today is CXL which adds cache coherency on top of pci-e.

Without cache coherency, you have to be more careful about how you use the memory and the performance story is complex. Ram over CXL is going to have worse perf than ram on the cpu memory controller, but there shouldn't be any big gotchas.