alt Hacker NewsCool! You might like our paper/demo on ray-tracing 1 TB-geometry scenes using thousands of AWS Lambda workers spun up just for the job: https://r2e2.dev/
This was led by my former PhD student Sadjad Fouladi who did a lot of creative work involving swarms of short-lived serverless functions for video encoding and distributed LLM inference (https://www.usenix.org/conference/nsdi17/technical-sessions/...) and software compilation (https://github.com/stanfordsnr/gg).
(I work on Hatchet)
This is really neat! Especially the global illumination that passes ray data between nodes, much fancier than what we've done here. You could imagine each component as an actor that receives messages via async channels (though perhaps it's more deterministic than that?).
I think bursting on serverless for massively parallelizable problems is a really interesting use-case. We were partially inspired for this demo by one of our users doing energy simulations at massive scale, who built a library called Scythe for this purpose: https://github.com/szvsw/scythe