The Tao of Backup

    The novice said: "I will save my working files, but not my system and application files, as they can be always be reinstalled from their distribution disks."
    The master made no reply.
    The next day, the novice's disk crashed. Three days later, the novice was still reinstalling software.

> The novice became suspicious and said: "Master, is all this 'Tao of Backup' stuff just so you can sell more copies of Veracity?"

> The master said: "Now you are truly enlightened."

> The master said, "Do not despair, for yesterday I took one of your backup tapes and posted it to my brother in China. He will return it."

The master should not have done this, even by sea mail, for the very first law of backup is: you will only learn to make proper backups after you suffer a catastrophic loss of data.

Protecting the student from this is depriving him of the only true knowledge, the one that is not merely stored in the head, but burned into the skin.

  He loaded the week-old backup, but it was the same. Eventually, he realized that the virus had struck four months before. He returned to the master and said: "Master, you have the only uncorrupted copy!"

  But the master had already overwritten it with a copy of DOOM.

I miss the World Wide Web, man...

For reference: a comprehensive backup + security plan for individuals https://nau.github.io/triplesec/

Great site - and lessons are still valid.

How you can tell it's from 1997:

The master fell silent, but on his way out he pocketed a 20 Gigabyte corporate backup tape that was lying around.

I just launched a music backup tool and service.

Storage is Hetzner Storage Box which so far feels great. Cheap and supports SSH SFTP WebDAV, so you can use rsync or restic and browse artifacts over www.

It feels a little slow hosting in Germany but using from US. But I haven’t found a provider with the same cost / features.

Any alternatives folks use for backups?

https://deadca7.com/blog/backups

Haven't seen that in ages and thought it had moved. 20GB tape containing all the corporate goodies. That makes me smile.

After reading the title, but before visiting the link, thought this was a backup of all of Terrance Taos work.

Not to be confused with the Zen of Backup…

A novice asked the master: “What must I do to learn the Zen of Backup?” The master whacked the novice in the head, grabbed the keyboard, and ran sudo rm -rf /. The novice smiled and left feeling enlightened.

After six years on my old build, last week I built a new computer. One of the things I invested in was treating the backup regime as the most important aspect of the new build. There's a piece of mind one gets from achieving the "seven heads"

This was a true joy to read and as pleasant as ads can come.

I felt old seeing this posted again, but I think I feel even older due to the lack of comments saying "I remember this"!

I assume that this was a [very good] ad for Veracity.

And the cautious reader asked:

  Surely it's not a plug for some product ?

  Bah, of course, it is !

I would expect the Tao of backup to loose all data every so often. What is success without failure?

Wow,.. this is so old that they still use the word "Tao" instead of "Dao". Still good advice, too.

Nice Epilogue

I like that the site is simple text (if not plain writing), but even in 1997 this multi-page layout would've a pain to read vs. a single page.

> But the master had already overwritten it with a copy of DOOM.

A master after my own heart.

I read all the steps, but not yet enlightened.

Loss of hard disk is a great way to declutter your system just saying

the backup that can be named is not the eternal backup

Obligatory link to the story about how Toy Story 2 got deleted.

https://www.youtube.com/watch?v=8dhp_20j0Ys

PS If you work in DevOps, SRE or Storage, this video may give you a panic attack

I accidentally let qwen wipe my 60tb nas the other day, and the tao of that is "well at least i dont have to stress about what im doing with my 30 years of rando .docs, facebook pics, games, ebooks and star trek episodes anymore"

c'est la vie, it was only 2tb filled and mostly games

Like giving a 6yo unix jr eng a crayon and root login. "you made them ALL apfs at the same time??" sorry dad. i can't believe people use this in prod, but i know i'm also not flagging perms my pi is ready to roll! gotta watch qwen3.6, they will be getting demoted.

I spent lots of time thinking about backups. I've got files dating back to 1991 and even some from the 80s (but I don't care about those so didn't bother backing them up: when the last 5"1/4 shall stop reading, those will be gone and it's fine).

I've even got stuff like copies of early websites made by friends, websites long gone and I'm pretty sure I'm the only one to still have backups of those.

To me verifying the backup should be part of the backup procedure itself and so I did just that: my backup procedure does verify that the backups can be decrypted/unarchived and it then verifies the files inside the backup.

Now the thing is: you don't need to verify 100% of your backups all the time (as in I don't decrypt and verify the checksums of 100% of my backups all the time). You can do random sampling: over n backups, you can be reasonably sure at least x% of the files are correct.

So random sampling is part of my backup procedure.

As those are encrypted backups, the verification also ensures that the backups can be decrypted (would be too bad otherwise, wouldn't it?).

I also believe the backuping procedure should not be able to go wild and destroy data. So my backup procedure is done by a podman container that access the volume with all my data (the one that needs to be backed up) read-only.

> To believe in one's backups is one thing. To have to use them is another."

Yes, which is why a proper backup procedure verifies the backup it just created. The greenlight is only given to the backup that's just been made once it's been verified by my verification script.

Some stuff can be verified automatically: for example say you backup a Git repo: a strict, full, git fsck on the backup (that is: on the Git repo once pulled out of the backup, before greenlighting it) works fine.

For other things I've got my own verifications.

> These hundreds of corrupted files have been flowing through my backup system. Now I do not know which files are clean and which are not.

Which is why many of my files, which I know aren't supposed to ever change anymore, have a partial checksum added as part of the filename.

I don't have:

    DSC0983747.JPG

    DSC0983747-b3-7e228491a0.JPG

Then it become very complicated for "something" (bit rot or malicious) to silently corrupt my stuff for the backuping procedure (which only has read-only access to data, remember) goes crazy bonkers and warns me as soon as two identically named files (one on the last backup and the current version) have an identical name but different content and one doesn't match the checksum (this is cause for a "stop the world" and immediate enquiry: and, yup, it already allowed me to catch a corrupted file).

I've moreover got a sheet of paper, laminated, that explains how to access the backups and that explanation is saved in many places (safe at the bank, safe at my brother's house in another country, etc.): should say my place burn with me inside, but my wife be safe, she'd be able to access the backups too.

As for my main data, it's RAID on ZFS on a machine with ECC and that's where the backups are made (and automated) from.

Proper backup procedure is not hard: it just takes some time to set up properly, once. Then it's done for a lifetime.

[flagged]

[flagged]

HTTPS cert is invalid