Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
scripts:backups_and_data [2022-06-03 20:42] – Tony | scripts:backups_and_data [2023-03-27 03:12] (current) – Tony | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Backups and Data ====== | ||
+ | |||
+ | |||
+ | ===== Backups ===== | ||
+ | |||
+ | ==== Borg ==== | ||
+ | |||
+ | [[scripts: | ||
+ | |||
+ | ===== Parity ===== | ||
+ | |||
+ | [[scripts: | ||
+ | |||
+ | ===== Compression ===== | ||
+ | |||
+ | ==== XZ/ZSTD ==== | ||
+ | |||
+ | |||
+ | I used to use XZ for compressing tar files. As of 2021-01, I have switched to ZStandard since it offers the same or slightly better ratio with -22, with MUCH faster decompression. | ||
+ | |||
+ | I will try to do a larger writeup and comparison in the future. | ||
+ | |||
+ | For compressing large amounts of files for archiving, where I won't need frequent random access, I use zpaq -m4. m5 is better but also takes forever. ZPAQ is particularly good if there are lots of duplicates. | ||
+ | |||
+ | For folders with large duplicates I rarely need, I am testing out dwarfs, since it hits a good ratio with -l7 (which uses zstd -22) and allows random access. Anything above -7 segfaults for me. | ||
+ | |||
+ | |||
+ | ==== ZPAQ ==== | ||
+ | |||
+ | If you are low on space and don't care about CPU time, it's a journaled archive. Very good compression ratio. | ||
+ | |||
+ | -m3 works fine for maybe-already compressed media like photos and videos you don't want to spend a year on. -m4 is great for more text based files. | ||
+ | |||
+ | zpaq a files.zpaq Documents/ -m4 | ||
+ | | ||
+ | -m5 takes an eternity, and is technically better but generally not worth it. (Bonus points for the RAM usage on a 16c32t system...) | ||
+ | |||
+ | ==== Dwarfs and Squashfs ==== | ||
+ | |||
+ | These are read-only filesystems. you make them, and it's a compressed file, but it can be mounted. | ||
+ | |||
+ | Squashfs: pretty standard. used on many linux installer. similar ratio to tar. can use zstandard. not best radio, but deduplicated, | ||
+ | |||
+ | For backup, borg also allows mounting, is deduplicated and compressed. Sqsh is just a "read only image" equivalent with compression. | ||
+ | |||
+ | Dwarfs: found it on github (by mhx?). faster then zpaq, better then squashfs. have not vetted source code though. use at your own risk. it is a cool FS though. | ||
+ | |||