logoalt Hacker News

HumanOstrich05/01/20251 replyview on HN

Taking an LVM snapshot and then copying the sqlite database from that is sufficient to keep it from being corrupted, but you can have incomplete transactions that will be rolled back during crash recovery.

The problem is that LVM snapshots operate at the block device level and only ensure there are no torn or half-written blocks. It doesn't know about the filesystem's journal or metadata.

To get a consistent point-in-time snapshot without triggering crash-recovery and losing transactions, you also need to lock the sqlite database or filesystem from writes during the snapshot.

    PRAGMA wal_checkpoint(FULL);
    BEGIN IMMEDIATE;  -- locks out writers
    .   /* trigger your LVM snapshot here */
    COMMIT;
You can also use fsfreeze to get the same level of safety:

    sudo fsfreeze -f /mnt/data      # (A) flush dirty pages & block writes
    lvcreate -L1G -s -n snap0 /dev/vg0/data
    sudo fsfreeze -u /mnt/data      # (B) thaw, resume writes
Bonus - validate the snapshotted db file with:

    sqlite3 mydb-snapshot.sqlite "PRAGMA integrity_check;"

Replies

remram05/02/2025

What does "losing transactions" mean? Some transactions will have committed before your backup and some will have committed after and therefore won't be included. I don't see the problem you are trying to solve?

Whether a transaction had started and gets transparently rolled back, or you had prevented from starting, what is the difference to you? Either way, you have a point-in-time snapshot, that time is the latest commit before the LVM snapshot.

You're discussing this in terms of "safety" and that doesn't seem right to me.

show 1 reply