logoalt Hacker News

mxmlnknyesterday at 9:24 PM1 replyview on HN

https://github.com/martinellimarco/indexed_zstd

https://github.com/martinellimarco/libzstd-seek

Note, however, that this can only seek to frames, and zstd still only creates files containing a single frame by default. pzstd did create multi-frame files, but it is not being developed anymore. Other alternatives for creating seekable zstd files are: zeekstd, t2sz, and zstd-seekable-format-go.


Replies

a_t48yesterday at 9:49 PM

Thanks, this is helpful. I might just end up using content defined chunking in addition/instead, but it's good to know that there is a path forward if I stick with the current architecture.