logoalt Hacker News

kianN • yesterday at 10:36 PM • 2 replies • view on HN

> When you write data to an actor’s SQLite database, the write is acknowledged once SSDs in 2 of 3 availability zones have it. Then, the data is batched to S3 in the background. This means that S3 is never on the write path, which lowers latency and cost. When reading data or waking an actor, Rivet reads from SSDs if already cached and falls back to S3 if not. SSD caches can be tuned to meet your needs, giving you configurable tiered storage for your actors.

As someone who is not incredibly deep in the agent space, but works a lot on scientific and distributed computing, this really caught my eye. Seems like this is primarily part of the cloud offering, but a true posix compatible filesystem that interops cleanly with S3 on the backend would be an incredibly interesting product (especially if it were open-sourced).

A brief sidenote: I did a triple take when I saw the name because Rivet is the internal name of our long-running job queue service. You all have great taste.


Replies

whalesalad • today at 5:17 AM

Lots of ways to use S3 as a filesystem. FUSE is one. AWS even offers it as a service now.

➕ show 1 reply
patcon • yesterday at 10:51 PM

I hate to be that guy, but celld does essentially the same thing, no? It waits until persisted to another node OR stored in S3, but unless something has gone horribly wrong, this will essentially always be replicating to the other node first. (Admittedly, I recall it used to be just replicating to 1 node was considered stored, and if that went down it could be lost. I think it's two now, or at least the bug is filed. Prob how the write time was so low in celld, but perhaps wasn't sustainable)

Regardless, glad for more stateful serverless options, even if I am partial to what CloudFlare folks build :)

➕ show 1 reply