mirror_zfs/cmd
Rob Norris f4aeb23f52 ddt: add "flat phys" feature
Traditional dedup keeps a separate ddt_phys_t "type" for each possible
count of DVAs (that is, copies=) parameter. Each of these are tracked
independently of each other, and have their own set of DVAs. This leads
to an (admittedly rare) situation where you can create as many as six
copies of the data, by changing the copies= parameter between copying.
This is both a waste of storage on disk, but also a waste of space in
the stored DDT entries, since there never needs to be more than three
DVAs to handle all possible values of copies=.

This commit adds a new FDT feature, DDT_FLAG_FLAT. When active, only the
first ddt_phys_t is used. Each time a block is written with the dedup
bit set, this single phys is checked to see if it has enough DVAs to
fulfill the request. If it does, the block is filled with the saved DVAs
as normal. If not, an adjusted write is issued to create as many extra
copies as are needed to fulfill the request, which are then saved into
the entry too.

Because a single phys is no longer an all-or-nothing, but can be
transitioning from fewer to more DVAs, the write path now has to keep a
copy of the previous "known good" DVA set so we can revert to it in case
an error occurs. zio_ddt_write() has been restructured and heavily
commented to make it much easier to see what's happening.

Backwards compatibility is maintained simply by allocating four
ddt_phys_t when the DDT_FLAG_FLAT flag is not set, and updating the phys
selection macros to check the flag. In the old arrangement, each number
of copies gets a whole phys, so it will always have either zero or all
necessary DVAs filled, with no in-between, so the old behaviour
naturally falls out of the new code.

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
Co-authored-by: Don Brady <don.brady@klarasystems.com>
Sponsored-by: Klara, Inc.
Sponsored-by: iXsystems, Inc.
Closes #15893
2024-08-16 12:02:39 -07:00
..
raidz_test RAID-Z expansion feature 2023-11-08 10:19:41 -08:00
zdb ddt: add "flat phys" feature 2024-08-16 12:02:39 -07:00
zed zed: Add deadman-slot_off.sh zedlet 2024-05-29 10:46:41 -07:00
zfs JSON output support for zpool get 2024-08-06 12:47:00 -07:00
zinject Parallel pool import 2024-04-22 09:42:38 -07:00
zpool JSON: Fix class values for mirrored special vdevs 2024-08-06 12:47:58 -07:00
zpool_influxdb Do not report bytes skipped by scan as issued. 2023-06-30 08:47:13 -07:00
zstream zstream: remove duplicate highbit64 definition 2024-08-09 14:31:41 -07:00
arc_summary Once more refactor arc_summary output 2024-08-05 09:22:51 -07:00
arcstat.in Fix arcstats for FreeBSD after zfetch support 2024-04-29 13:28:50 -07:00
dbufstat.in Consider dnode_t allocations in dbuf cache size accounting 2023-11-17 13:25:53 -08:00
fsck.zfs.in cmd: move single-file binaries up, extract udev programs to udev/ 2022-05-10 10:20:34 -07:00
Makefile.am config: use -Wno-format-truncation globally 2024-02-26 12:23:55 -08:00
mount_zfs.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
zfs_ids_to_path.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zgenhostid.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zhack.c Provide macros for setting and getting blkptr birth times 2024-03-25 15:01:54 -07:00
zilstat.in zil: add stats for commit failure/fallback (#16315) 2024-07-25 16:53:59 -07:00
ztest.c ddt: add support for prefetching tables into the ARC 2024-07-26 09:16:18 -07:00
zvol_wait zvol_wait logic may terminate prematurely 2022-10-11 12:12:04 -07:00