logoalt Hacker News

jamesblonde • today at 10:21 AM • 1 reply • view on HN

Git was designed with assumptions about local file storage which make it challenge when storing data in a distributed file system. We built our distributed filesystem, HopsFS, to store data in S3. To support high performance git in FUSE, our writes hit remote NVMe disk(s) and asynchronously sync to S3 (you can replicate on NVMes or just do failure recovery). Previously, we used remote NVMe disks as a write-through cache, but that doesn't work with git, S3 latency is too high.


Replies

EngineeringStuf • today at 11:56 AM

We built similar at Swarmfile, and it's compatible with Git too