Improved Deduplication Efficiency in OpenZFS

iXsystems and Klara are collaborating to implement Fast Dedup support in OpenZFS, with plans to integrate these improvements into TrueNAS SCALE starting next month.

The upgraded OpenZFS Fast Dedup brings a significant overhaul to the file system’s deduplication capabilities. Notable improvements include addressing challenges related to deduplication hash tables, ensuring they are in memory when needed, and tackling performance and scalability issues. The new iteration allows dynamic sizing of metadata to fit in RAM or dedicated flash devices. The metadata structure has undergone a re-engineering process, and the deduplication table now prioritizes dedup-able data while efficiently pruning blocks with no deduplication potential.

For further insights into the OpenZFS Fast Dedup feature, refer to the iXsystems blog.

Source: Phoronix.