alt Hacker NewsI am building a cloud
Comments
Potentially useful context: OP is one of the cofounders of Tailscale.
> Traditional Cloud 1.0 companies sell you a VM with a default of 3000 IOPS, while your laptop has 500k. Getting the defaults right (and the cost of those defaults right) requires careful thinking through the stack.
I wish them a lot of luck! I admire the vision and am definitely a target customer, I'm just afraid this goes the way things always go: start with great ideals, but as success grows, so must profit.
Cloud vendor pricing often isn't based on cost. Some services they lose money on, others they profit heavily from. These things are often carefully chosen: the type of costs that only go up when customers are heavily committed—bandwidth, NAT gateway, etc.
But I'm fairly certain OP knows this.
There's a common conversation that goes on around AI: some people swear its a complete waste of time and total boondoggle, some that its a good tool when used correctly, and others that its the future and nothing else matters.
I see the same thing happen with Kubernetes. I've run clusters from various sizes for about half a decade now. I've never once had an incident that wasn't caused by the product itself. I recall one particular incident where we had a complete blackout for about an hour. The people predisposed to hating Kubernetes did everything they could to blame it all on that "shitty k8s system." Turns out the service in question simply DOS'd by opening up tens of thousands of ports in a matter of seconds when a particular scenario occurred.
I'm neither in the k8s is the future nor k8s is total trash. It's a good system for when you genuinely need it. I've never understand the other two sides of the equation.
The point about VMs being the wrong shape because they’re tied to CPU/memory resonates hard. The abstraction forces you to pay for time, not work.
I ended up buying a cheap auctioned Hetzner server and using my self-hostable Firecracker orchestrator on top of it (https://github.com/sahil-shubham/bhatti, https://bhatti.sh) specifically because I wanted the thing he’s describing — buy some hardware, carve it into as many VMs as I want, and not think about provisioning or their lifecycle. Idle VMs snapshot to disk and free all RAM automatically. The hardware is mine, the VMs are disposable, and idle costs nothing.
The thing that, although obvious, surprised me most is that once you have memory-state snapshots, everything becomes resumable. I make a browser sandbox, get Chromium to a logged-in state, snapshot it, and resume copies of that session on demand. My agents work inside sandboxes, I run docker compose in them for preview environments, and when nothing’s active the server is basically idle. One $100/month box does all of it.
> Agents, by making it easiest to write code, means there will be a lot more software. Economists would call this an instance of Jevons paradox. Each of us will write more programs, for fun and for work.
There is already so much software out there, which isn't used by anyone. Just take a look at any appstore. I don't understand why we are so obsessed with cranking out even more, whereas the obvious usecase for LLMs should be to write better software. Let's hope the focus shifts from code generation to something else. There are many ways LLMs can assist in writing better code.
Nice post. exe.dev is a cool service that I enjoyed.
I agree there is opportunity in making LLM development flows smooth, paired with the flexibility of root-on-a-Linux-machine.
> Time and again I have said “this is the one” only to be betrayed by some half-assed, half-implemented, or half-thought-through abstraction. No thank you.
The irony is that this is my experience of Tailscale.
Finally, networking made easy. Oh god, why is my battery doing so poorly. Oh god, it's modified my firewall rules in a way that's incompatible with some other tool, and the bug tracker is silent. Now I have to understand their implementation, oh dear.
No thank you.
Numerous people are denigrating DevOps people - resume padding, over-complexity, etc.
I think that's startup-thinking, at least in my experience. Maybe in a small company the DevOps guy does all infra.
In my experience, especially in financial services, who runs the show are platform engineering MDs - these people want maximum control for their software engineers, who they split up into a thousand little groups who all want to manage their own repos, their own deployments, their own everything. It's believed that microservices gives them that power.
I guarantee you devops people hate complexity, they're the ones getting called at night and on the weekend, because it's supposedly always an "infrastructure issue" until proven otherwise.
Also the deployment logs end up in a log aggregation system, and god forbid software developers troubleshoot their own deployments by checking logs. It's an Incident.
Are microservices a past fad yet?
i just use Hetzner.
Everything which cloud companies provide just cost so much, my own postgres running with HA setup and backup cost me 1/10th the price of RDS or CloudSQL service running in production over 10 years with no downtime.
i directly autoscales instances off of the Metrics harvested from graphana it works fine for us, we've autoscaler configured via webhooks. Very simple and never failed us.
i don't know why would i even ever use GCP or AWS anymore.
All my services are fully HA and backup works like charm everyday.
Virtual machines are the wrong abstraction. Anyone who has worked with startups knows that average developers cannot produce secure code. If average developers are incapable of producing secure code, why would average non-technical vibe-coders be able to? They don't know what questions to ask. There's no way vibe coders can produce secure backend software with or without AI. The average software that AI is trained on is insecure. If the LLM sees a massive pile of fugly vibe-coded spaghetti and you tell it "Make it secure please", it will turn into a game of Whac-a-Mole. Patch a vulnerability and two new ones appear. IMO, the right solution is to not allow vibe-coders to access the backend. It is beyond their capabilities to keep it secure, reliable and scalable, so don't make it their responsibility. I refuse to operate a platform where a non-technical user is "empowered" to build their own backend from scratch. It's too easy to blame the user for building insecure software. But IMO, as a platform provider, if you know that your target users don't have the capability to produce secure software, it's your fault; you're selling them footguns.
There are plenty of alternatives out there. I built https://shellbox.dev, which gives you instant vms via ssh where unlike exe you pay only for what you use-- scale to zero. It is also regular linux, supporting vscode and zed remote, Nested virtualization, etc.
If you're looking to invest im fine with only $5M :)
Shameless plug: https://clawk.work/
`ssh you/repo/[email protected]` → jump directly into Claude Code (or Codex) with your repo cloned and credentials injected. Firecracker VMs, 19€/mo.
POC, please be kind.
People who don't understand the software bloat cycle are doomed to repeat it.
Lean software -> missing features users want -> add features over time -> bloated mess -> we need a smaller rewrite -> Lean software -> ...
all of the grievances resulting in this move is a simple outcome of the cost of convenience. but it should not need going full opposite end to get something good enough.
dedicated servers, as hinted by others here, addresses the vast majority of issues one may face for any non-enterprise needs. if you know about IOPS and care about them, odds are that running a simple open-source project [1] on top of one is all you need to do to move on with your day.
need redundancy, etc.? can complement with another one in another provider/region or put CF in front of your box. this is clearly working well enough for some of the commenters who are able to sell their own service on top of this approach.
That's insane funding so congrats.
Just shows I'm the Dropbox commentator. I have what exe provides on my own and am shocked by the value these abstractions provide everyone else!! One off containers on my own hardware spin up spin down run async agents, etc, tailscale auth, team can share or connect easily by name.
Europe is crying out for sovereign clouds. If this is to be a viable alt cloud, US jurisdiction is a no.
Not sure we can move away from cpu/memory/io budgeting towards total metal saturation because code isn't what it used to be because no one handles malloc failure any more, we just crash OOM
This is the problem for me with the cloud:
> Finally, clouds have painful APIs. This is where projects like K8S come in, papering over the pain so engineers suffer a bit less from using the cloud. But VMs are hard with Kubernetes because the cloud makes you do it all yourself with lumpy nested virtualization. Disk is hard because back when they were designing K8S Google didn’t really even do usable remote block devices, and even if you can find a common pattern among clouds today to paper over, it will be slow. Networking is hard because if it were easy you would private link in a few systems from a neighboring open DC and drop a zero from your cloud spend. It is tempting to dismiss Kubernetes as a scam, artificial make work designed to avoid doing real product work, but the truth is worse: it is a product attempting to solve an impossible problem: make clouds portable and usable. It cannot be done.
Please learn from Unix's mistakes. Learn from Nix. Support create-before-destroy patterns everywhere. Forego all global namespaces you can. Support rollbacks everywhere.
If any cloud provider can do that, cloud IaC will finally stop feeling so fake/empty compared to a sane system like NixOS.
Ok, what am I missing? exe.dev says: "$20/month for your VMs One price, no surprises. You get 2 CPUs, 8 GB of RAM, and 25 GB of disk".
Fine, their UI is different, but I don't see any real difference from other providers.
I really like exe.dev's pricing model where I pay a fixed monthly fee for compute and then can split it up into as many VMs as I want. I use exe.dev to run little vibe-coded apps and it's nice to just leave them running without a spend meter ticking up.
We're thinking about switching to this pricing model for our own startup[1] (we run sandboxed coding agents for dev teams). We run on Daytona right now for sandboxes. Sometimes I spin up a sandboxed agent to make changes to an app, and then I leave it running so my teammate can poke around and test the running app in the VM, but each second it's running we (and our users) incur costs.
We can either build a bunch of complicated tech to hibernate running sandboxes (there's a lot of tricky edge cases for detecting when a sandbox is active vs. should be hibernated) or we can just provision fixed blocks of compute. I think I prefer the latter.
Hahaha! Have fun! I‘m doing the same - together with Claude Code. Since August. With https (mTLS1.3) everywhere, because i can. Just my money, just my servers, just for me. Just for fun. And what a fun it is!
> The standard price for a GB of egress from a cloud provider is 10x what you pay racking a server in a normal data center.
Oh, that’s too kind. More like 100x to 1000x. Raw bandwidth is cheap.
I relate to the points about obscure platform limits, and leaky abstractions, but when I look at the exe.dev platform, it might be the most obscure PaaS I've seen, and has it's own strange abstractions.
The shell command to start a new vm, has a --prompt flag to get an LLM to configure the VM for you.
VM's have no public ipv4 IP, and the ipv6 IP doesn't seem to allow incoming connections.
The only supported inbound connections are via their HTTP proxy.
There is no private networking.
At first I interpreted the complaint about cloud providers not offering nested-virtualization, as something he intends to address by offering it as a feature, but no, instead he means that exe.dev's VM abstraction eschews the need for it.
I'm not sure if this is the direction the OP is going, but I would love to see a world where local small-time investors can get a bank loan, rent a facility, set up a bunch of computers, and run open-source cloud software on them that provides 95% of the features that most businesses need.
Running a cloud data center could be a business like operating a self-storage facility or a car wash. Small investors love this kind of operation.
This looks like an excellent platform for running a "homelab" in the cloud (no, the irony is not lost on me) for lighter stuff like Readeck, Calibre-web, Immich. Maybe even Home Assistant too if we can find a way (Tailscale?) to get the mDNS/multicast traffic tunnelled.
I'd like write program / run program / debug program to be as easy as it is in roblox. It isn't that easy though, the set of things you need to do it well is extensive. I wouldn't be averse to a new platform, one in which all io is over highly performant queues, but the moras of existing software tied to unix is large, just look at compilers and all the child processes they launch. It was always shims and it will always be shims.
I have had an eye on this for a while (found via pi.dev) but I don't really have a solid use case for it, but the idea/concept of is appealing where the price is not. I can buy a £100-150 mini-pc with better hardware to run 24/7 for my own VMs extending my homelab (granted my ISP doesn't put any restrictions on me, I know many others can't say the same).
You can see their base docker image here - https://github.com/boldsoftware/exeuntu
In classic soft-launch style, their blog is quite good. I really liked reading how they're doing ssh forwarding similar to http reverse proxies using the 'host' header [0]
I use kubernetes extensively at work. I don't manage the kubernetes cluster anymore since now we have a team that runs centralized services and you can request a namespace with a quota. But back when my team had a dedicated Azure Kubernetes cluster it was not that bad as people says it is and the biggest hassle was the extremely short lived support for each version.
Then I started to realize most people who complain are rolling their own which is also not bad since there are products like k3s that are very simple to use.
It seems things start to fall apart when they try to stuff it with all kinds of crazy idiotic controllers and the favorite of the month CNI and CSI. I always shake my head when I see people creating sand castles by setting up stuff like Ceph from within the cluster.
If you want to play with it keep things simple and have all the persistent data outside of the cluster. Use good old NFS instead of the latest longceph horngluster version. Keep databases and the container registry out. Treat it like a compute pool not a virtual datacenter. Stop recursing chickens inside eggs.
I'm excited to see what they put together, because this raises a number of similar gripes I have with public cloud in its current state:
* Insistence on adding costly abstractions to overcome the limitations of non-fungible resources
* Deliberate creation of over or under-sized resource "pieces" instead of letting folks consume what they need
* Deliberate incompatibility with other vendors to enforce lock-in
I pitched a "Universal Cloud" abstraction layer years ago that never got any traction, and honestly this sounds like a much better solution anyhow. When modern virtualization is baked into OS kernels, it doesn't make a whole lot of sense to enforce arbitrary resource sizes or limits other than to inflate consumption.
Kubernetes without all the stuff that makes it a bugbear to administrate, in other words. Let me buy/rent a pool of stuff and use it how I see fit, be it containers or VMs or what-have-you.
Lots of negativity towards k8s in here. It's always funny to me when $WILDLY_POPULAR_TECH gets ripped apart like this, as though no one has ever had a positive experience with it. I've seen similar pile-ons for React, microservices, git, PHP, JavaScript, cloud services, really anything that's been adopted at scale.
I have mixed feelings about this concept, I agree that the way clouds work now is far from great and stronger abstractions are possible. But this article offers nothing of the sort, it just handwaves 'we solve some problem and that saves you tokens'???
Checking the current offering, it's just prepaid cloud-capacity with rather low flexibility. It's cheap though, so that is nice I guess. But does this solve anything new? Anything fly.io orso doesn't solve?
What is the new idea here? Or is it just the vibes?
exe.dev. 111 IN A 52.35.87.134
52.35.87.134 <- Amazon Technologies Inc. (AT-88-Z)
I love this and agree with it almost in entirety. One thing that would be good to retain is 'regions' e.g 3 DCs under 10KM apart linked with free/very low cost internal network. It is cheap to build in colo these days with the advent of campus XCs and ever lowering cost of DF, optics and 400G/800G switches.
As a happy customer, godspeed to exe.dev.
I've found the quality and simplicity to be an attractive solution for lazy devops when I need to reach for a second computer
I have trouble seeing how this is different to linode, if i invest time in a new VM api, this has to work for cloud or my own machines transparently. Lastly as much as i share the disappointment in k8s promise, this seems a bit too simple, there is a reason homelabs mostly standardised on compose files.
Hilariously, they have this linked (“That must be worst website ever made.”: https://news.ycombinator.com/item?id=46399903) under What people are saying.
It seems really cool, but the entry-level tier just seems too expensive. I can get a single pain-in-the-ass OVH VPS for $7. I just need something better than that for the same price.
This makes me wonder if I could get a few million in funding to rent out some Oxide racks. I'd love to touch some Oxide hardware and this seems like a good way to do it.
That's really cool!
One thing I'm confused with is how to create a shared resources like e.g. a redis server and connect to it from other vms? It looks now quite cumbersome to setup tailscale or connect via ssh between VMS. Also what about egress? My guess is that all traffic billed at 0.07$ per GB. It looks like this cloud is made to run statefull agents and personal isolated projects and distributed systems or horizontal scaling isn't a good fit for it?
Also I'm curious why not railway like billing per resource utilization pricing model? It’s very convenient and I would argue is made for agents era.
I did setup for my friends and family a railway project that spawns a vm with disk (statefull service) via a tg bot and runs an openclaw like agent - it costs me something like 2$ to run 9 vms like this.
You should do it in Europe, so much demand for European clouds and very weak offerings.
Comparing laptop SSD to cloud network drive is misleading.
EC2 provides the *d VMs that have SSDs with high IOPS at much lower cost than network SSDs. They are ephemeral, but so is laptop and its SSD - it can loose the data. From AWS docs "If you stop, hibernate, or terminate an instance, data on instance store volumes is lost.".
The "one price" is oddly small for a cloud company. I'm sure it's nice and fast but the $20/mo seems smaller than some companies' free tiers, especially for disk.
The main reason clouds offer network block devices is abstraction.
Half the work of pricing a solo SaaS is modeling what a bad month looks like if egress or IOPS spike. You end up pricing defensively to protect against your own infra bill instead of pricing to the value you provide. A fixed bucket of compute with clear limits is way easier to build a business on than a meter that could run anywhere.
I don’t get it.
Starting a digital ocean droplet is a single curl call. Starting a hetzner server is as well. Their api’s are completely fine and known to llm’s.
Why would agents learn exe’s way of setting up / deploying / binding to ports / auth, rather than just ssh’ing into a vm..?
I think I am interested in this? I run a bunch of small web apps, currently as fly.io machines. I love fly, but it adds up when I have a bunch of small things that I want isolated — I wish I could have even smaller Fly instances. Exe.dev seems like a good middleground where I can allocate the compute from tiny to large. (?)
I can't see why I would want this, but I do love Tailscale so I'm excited to see what new stuff he comes up with here.
I'm still new to cloud computing. I've only ever used linode. What is this supposed to be? I couldn't figure out a specific design through the article well. Pls help
This is being accurately called "cloud for developers". If it were for enterprise, it should cost 1000x to create thousands of positions, multiple VPs, executives, etc with a bill in 100s of millions of dollars. Execs wants high capex/opex and a massive headcount. BIGG numbers mean bigger titles and compensation.
With LLMs there is no real dev velocity penalty of using high perf. langs like say Rust. A pair of 192 Core AMD EPYC boxes will have enough headroom for 99.9% of projects.
I will follow this one for sure. There are a few more companies with the extremely ambitious goal of "a better AWS", and I am interested in the various strategies they take to approach that goal incrementally.
A service offering VMs for $20 is a long way from AWS, but I see how it makes sense as a first step. AWS also started with EC2, but in a completely different environment with no competition.
> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig.
So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined networking layer on top of it.
After spending a lot of time "optimizing" or "hardening" the cluster, cloud spend has doubled or tripled. Incidents have also doubled or tripled, as has downtime. Debugging effort has doubled or tripled as well.
I ended up saying goodbye to those devops folks, nuking the cluster, booted up a single VM with debian, enabled the firewall and used Kamal to deploy the app with docker. Despite having only a single VM rather than a cluster, things have never been more stable and reliable from an infrastructure point of view. Costs have plummeted as well, it's so much cheaper to run. It's also so much easier and more fun to debug.
And yes, a single VM really is fine, you can get REALLY big VMs which is fine for most business applications like we run. Most business applications only have hundreds to thousands of users. The cloud provider (Google in our case) manages hardware failures. In case we need to upgrade with downtime, we spin up a second VM next to it, provision it, and update the IP address in Cloudflare. Not even any need for a load balancer.