Tony Tascioglu Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Parity and Par2 ======


There are a few option. Par2 (parchive) seems to be the standard here. Of course ZFS and BTRFS can keep track of parity bits in RAID setups, and ZFS will automatically correct the bit errors from the parity drive (as long as you don't pull an LTT and periodically scrub the drives).

There is also parity mode with RAR and Dvdisaster, though I have limited experience with both. Par2 is kind-of inefficient with the overhead per file, and it was traditionally run on a single tar or zip, etc, not multiple files. Par2 supports multiple files (inefficiently). I still rather par2 a SquashFS or ZPAQ file with the space savings of those plus single file mode is more efficient.

==== Par2 larger files ====

When trying the multi file archive, I was annoyed that it would add the Digikam XMP sidecar files (used to avoid writing EXIF into videos and RAW photos), which is a waste of space.

So, instead of giving a path, you can give a list of filenames and filter the larger files (or by name like *.ARW or *.NEF).

  par2create -v -m4096 -b32768 -c1023 -t24 -R rec -- (find -type f -size +200k)
  
This is intended for fish shell, hence no $. This uses a max of 4G of memory, uses the maximum allowed 32768 blocks (meaning max 32k files), with 1023 parity blocks (allowing us to lose 1023 of the 32768 original blocks and still being able to recover). -t24 uses 24 threads, change this for your system. rec is the filename and -- marks subsequent arguments as file names (put a path here instead to just do every file).

When verifying, I also add -T1 to only do 1 file at a time. This is to avoid thrashing on my spinning rust hard drives. Might not matter on SSDs. It's especially important on optical media, where seeks are expensive, and the media is best treated as a linear archive like tape. (Yes it's random access but that laser carriage is slow vs a HDD head and it's a linear spiral groove anyway).