alt Hacker NewsNice post and technically impressive work. I agree we need to understand the build pipeline and be able to do things locally. However, depending on your electricity cost, it might not make sense financially. These old servers are not energy efficient at all (I'm guessing that old Xeon server will easily pull 200W on load), and that model is currently at 0.1$/0.3$ per 1M tokens (with 76 tps and 262k context) in Openrouter (also, these servers are LOUD).
EDIT: I stand corrected, 200W is apparently way too high of an estimate. I used to run a bunch of old Xeon servers and they slurped watts like crazy, but I can't remember which ones exactly those were.
Replies
It should be closer to 85W on load. And it's incredibly silent on even a low end cooler. I rarely get above 50° Celcius.
These servers are loud if you're trying to fit them into a 1U or 2U, which requires high speed fans to generate the necessary static pressure to push air through the case. I run a similar setup in a 4U case with slow 120mm fans and it's fine.
2620v4 is not a power slurping beast. Depending on the server board, it might not be either. Servers are often loud, but it depends.
There's a lot of budget hosting built around chips like these, and they're suprisingly power efficient.