Backups and Data

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Backups and Data ======


===== Backups =====

==== Borg ====

[[scripts:backups:borg|Borg]]

===== Parity =====

[[scripts:backups:parity_par2|Parity and Par2]]

===== Compression =====

==== XZ/ZSTD ====


I used to use XZ for compressing tar files. As of 2021-01, I have switched to ZStandard since it offers the same or slightly better ratio with -22, with MUCH faster decompression.

I will try to do a larger writeup and comparison in the future.

For compressing large amounts of files for archiving, where I won't need frequent random access, I use zpaq -m4. m5 is better but also takes forever. ZPAQ is particularly good if there are lots of duplicates.

For folders with large duplicates I rarely need, I am testing out dwarfs, since it hits a good ratio with -l7 (which uses zstd -22) and allows random access. Anything above -7 segfaults for me.


==== ZPAQ ====

If you are low on space and don't care about CPU time, it's a journaled archive. Very good compression ratio.

-m3 works fine for maybe-already compressed media like photos and videos you don't want to spend a year on. -m4 is great for more text based files.

  zpaq a files.zpaq Documents/ -m4
  
-m5 takes an eternity, and is technically better but generally not worth it. (Bonus points for the RAM usage on a 16c32t system...)

==== Dwarfs and Squashfs ====

These are read-only filesystems. you make them, and it's a compressed file, but it can be mounted.

Squashfs: pretty standard. used on many linux installer. similar ratio to tar. can use zstandard. not best radio, but deduplicated, and allows random reads since it can be mounted.

For backup, borg also allows mounting, is deduplicated and compressed. Sqsh is just a "read only image" equivalent with compression.

Dwarfs: found it on github (by mhx?). faster then zpaq, better then squashfs. have not vetted source code though. use at your own risk. it is a cool FS though.