scripts:files:deduplication

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

scripts:files:deduplication [2023-01-15 22:55] – created Tonyscripts:files:deduplication [2023-01-15 22:55] (current) – Tony
Line 1: Line 1:
 +====== Deduplicate Files on BTRFS ======
 +
 +BTRFS, ZFS, XFS, and many other file systems allow copy-on-write functionality, allowing multiple files to use the same blocks until one is updated.
 +
 +These is sometimes called "reflinks", **NOT** to be confused with hardlinks and symlinks/softlinks. (I'll make a video on this soon explaining the differences).
 +
 +While BTRFS doesn't have online deduplication, it has 2 ways of deduplicating:
 +
 +1. Programs such as rmlint can be used to convert identical files into reflinks, saving space.
 +
 +  sudo rmlint -c sh:link -g /path/to/deduplicate
 +
 +2. Programs like duperemove can link individual extents within files
 +
 +  sudo duperemove -r -d -v -h --hashfile duperemove /path/to/deduplicate
 +
 +
 +