alt Hacker NewsFrankly weird that they were resorting to high level containers for this in the first place. Also, this line struck me as odd
> Big Pineapple uses jemalloc, an allocator designed for multithreaded, allocation-heavy workloads.
jemalloc multithreaded performance is actually poor(ish) compared to other modern allocators, which makes it a weird choice. But even weirder is why they're even using an allocator in the first place compared to a va MAP_ANON | MAP_NORESERVE arena carveout approach? You can also do punning that way too, which I'm not even certain if Rust supports?
Replies
I would also have instinctively reached for a large VM reservation to exploit demand paging. I have used that pattern a lot in C++ but not in Rust, so I don't know how difficult it would be to implement there.
Rust supports punning via pointer casting, but you'll want to use #[repr(C)] on any data types used
An approach like that would be at constant war with the borrow checker in Rust. Apparently it is possible but there is enough friction that these guys went a different route.