These days there's usually two networks. Multi-trillion parameter models are too big to fit on one box of 8x H100's, so many of these are running RDMA (800Gb/sec on GB300 series) for rack-scale/many row-scale inference.

For GB300 series, pricing along the lines of $500k/server, 9 servers/rack ($4.5M/rack) before looking at water cooling, 800Gb RDMA network, and your 100Gb IP network