Differences

This shows you the differences between two versions of the page.

--- scripts:backups_and_data [2022-04-12 19:08] – Tony
+++ scripts:backups_and_data [2023-03-27 03:12] (current) – Tony
@@ Line 1: / Line 1: @@
+====== Backups and Data ======
+===== Backups =====
+==== Borg ====
+[[scripts:backups:borg|Borg]]
+===== Parity =====
+[[scripts:backups:parity_par2|Parity and Par2]]
+===== Compression =====
+==== XZ/ZSTD ====
+I used to use XZ for compressing tar files. As of 2021-01, I have switched to ZStandard since it offers the same or slightly better ratio with -22, with MUCH faster decompression.
+I will try to do a larger writeup and comparison in the future.
+For compressing large amounts of files for archiving, where I won't need frequent random access, I use zpaq -m4. m5 is better but also takes forever. ZPAQ is particularly good if there are lots of duplicates.
+For folders with large duplicates I rarely need, I am testing out dwarfs, since it hits a good ratio with -l7 (which uses zstd -22) and allows random access. Anything above -7 segfaults for me.
+==== ZPAQ ====
+If you are low on space and don't care about CPU time, it's a journaled archive. Very good compression ratio.
+-m3 works fine for maybe-already compressed media like photos and videos you don't want to spend a year on. -m4 is great for more text based files.
+  zpaq a files.zpaq Documents/ -m4
+-m5 takes an eternity, and is technically better but generally not worth it. (Bonus points for the RAM usage on a 16c32t system...)
+==== Dwarfs and Squashfs ====
+These are read-only filesystems. you make them, and it's a compressed file, but it can be mounted.
+Squashfs: pretty standard. used on many linux installer. similar ratio to tar. can use zstandard. not best radio, but deduplicated, and allows random reads since it can be mounted.
+For backup, borg also allows mounting, is deduplicated and compressed. Sqsh is just a "read only image" equivalent with compression.
+Dwarfs: found it on github (by mhx?). faster then zpaq, better then squashfs. have not vetted source code though. use at your own risk. it is a cool FS though.