logoalt Hacker News

cookiengineertoday at 4:35 PM2 repliesview on HN

The great part is that you can always build your own selfhosted tools. There is nothing that can't be done at home, it's just a calculation of how much you're willing to spend.

Lately though the RAM crisis is continuing and making things like this more unfeasible. But you can still use a lot of smaller models for coding and testing tasks.

Planning tasks I'd use a cloud hosted one, for now, because gemma4 isn't there yet and because the GPU prices are still quite insane.

The cool and fun part is that with ollama and vllm you can just build your own agentic environment IDE, give it the tools you like, and make the workflow however you like. And it isn't even that hard to do, it just needs a lot of tweaking and prompt fiddling.

And on top of that: Use kiwix to selfhost Wikipedia, stackoverflow and devdocs. Give the LLM a tool to use the search and read the pages, and your productivity is skyrocketing pretty quickly. No need anymore to have internet, and a cheap Intel NUC is good enough for self-hosting a lot of containers already.

Source: I am building my own offline agentic environment for Golang [1] which is pretty experimental but sometimes it's also working.

[1] https://github.com/cookiengineer/exocomp


Replies

xantronixtoday at 5:15 PM

I'm definitely all in on self-hosting, though I rent my compute and pay for bandwidth with Linode and storage with rsync.net.

The LLM bit though, personally, is just not for me.

aaztehcytoday at 9:19 PM

[flagged]