TrueNAS Highlights of OpenZFS Summit: Fast Dedup and RAIDZ Expansion

TrueNAS, a leading provider of storage systems, uses OpenZFS as the foundation for its data management layer. As the deployment vehicle for the majority of OpenZFS storage systems used today, TrueNAS is excited about the continuous improvement of OpenZFS.

The 11th annual OpenZFS Developer Summit for 2023 kicked off on October 16th in San Francisco. Among the many exciting projects being developed, two projects stand out with significant contributions and investments from iXsystems.

One of the primary issues with traditional deduplication (dedup) with ZFS has been the need to keep the dedup tables in memory at all times, which led to performance penalties and usability issues. However, with the introduction of Fast Dedup, the metadata size is now constrained to fit in either RAM or flash, eliminating the performance penalty. The metadata structure for Fast Dedup has been completely re-engineered to allow efficient updates and the ability to evict non-deduplicated blocks. These improvements, combined with properly configured storage, will result in a significant improvement in dedup performance for larger systems. Allan Jude, a major sponsor of the Fast Dedup project, will be presenting the details at the summit.

RAIDZ expansion is another exciting project that is entering its final quality assurance cycle. This feature allows a small pool with as few as two drives to be gradually expanded with one drive at a time, preserving existing data with its original parity. The administrative process for small systems has also been simplified. Don Brady and Matt Ahrens will be presenting the latest updates on RAIDZ expansion at the summit. This project has taken a few years but is now in its final stretch and will be included in OpenZFS 2.3.

The current release, OpenZFS 2.2, has been integrated into TrueNAS SCALE 23.10 (Cobia). The release candidate version of Cobia, which includes dRAID, has been successfully provided to the community with over 3,000 testers. The formal release of TrueNAS SCALE 23.10 is planned in the coming weeks. In early 2024, TrueNAS CORE 13.1 will be released with OpenZFS 2.2, and TrueNAS Enterprise appliances will also use OpenZFS 2.2 in their respective software versions.

OpenZFS 2.3 (or potentially 3.0) is expected to take most of the next 12 months to mature and reach release quality and status. The RAIDZ expansion and Fast Dedup features will be integrated with TrueNAS at that time. Early availability of Nightlies and BETA software is expected in mid-2024 for TrueNAS SCALE.

Source: TrueNAS.