mirror_zfs/cmd
Matthew Ahrens 6774931dfa
Extend zdb to print inconsistencies in livelists and metaslabs
Livelists and spacemaps are data structures that are logs of allocations
and frees.  Livelists entries are block pointers (blkptr_t). Spacemaps
entries are ranges of numbers, most often used as to track
allocated/freed regions of metaslabs/vdevs.

These data structures can become self-inconsistent, for example if a
block or range can be "double allocated" (two allocation records without
an intervening free) or "double freed" (two free records without an
intervening allocation).

ZDB (as well as zfs running in the kernel) can detect these
inconsistencies when loading livelists and metaslab.  However, it
generally halts processing when the error is detected.

When analyzing an on-disk problem, we often want to know the entire set
of inconsistencies, which is not possible with the current behavior.
This commit adds a new flag, `zdb -y`, which analyzes the livelist and
metaslab data structures and displays all of their inconsistencies.
Note that this is different from the leak detection performed by
`zdb -b`, which checks for inconsistencies between the spacemaps and the
tree of block pointers, but assumes the spacemaps are self-consistent.

The specific checks added are:

Verify livelists by iterating through each sublivelists and:
- report leftover FREEs
- report double ALLOCs and double FREEs
- record leftover ALLOCs together with their TXG [see Cross Check]

Verify spacemaps by iterating over each metaslab and:
- iterate over spacemap and then the metaslab's entries in the
  spacemap log, then report any double FREEs and double ALLOCs

Verify that livelists are consistenet with spacemaps.  The space
referenced by livelists (after using the FREE's to cancel out
corresponding ALLOCs) should be allocated, according to the spacemaps.

Reviewed-by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Sara Hartse <sara.hartse@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
External-issue: DLPX-66031
Closes #10515
2020-07-14 17:51:05 -07:00
..
arc_summary Centralize variable substitution 2020-07-14 17:33:44 -07:00
arcstat Centralize variable substitution 2020-07-14 17:33:44 -07:00
dbufstat Centralize variable substitution 2020-07-14 17:33:44 -07:00
fsck_zfs Fix typos in cmd/ 2019-08-30 09:43:30 -07:00
mount_zfs Use abs_top_builddir when referencing libraries 2020-07-10 14:26:32 -07:00
raidz_test Unconditionally enable debugging for libzpool 2020-07-10 15:30:31 -07:00
vdev_id Fix typos in cmd/ 2019-08-30 09:43:30 -07:00
zdb Extend zdb to print inconsistencies in livelists and metaslabs 2020-07-14 17:51:05 -07:00
zed Centralize variable substitution 2020-07-14 17:33:44 -07:00
zfs Remove dependency on sharetab file and refactor sharing logic 2020-07-13 09:19:18 -07:00
zfs_ids_to_path Use abs_top_builddir when referencing libraries 2020-07-10 14:26:32 -07:00
zgenhostid bash scripts: use /usr/bin/env for bash shebangs 2020-02-10 13:13:46 -08:00
zhack Unconditionally enable debugging for libzpool 2020-07-10 15:30:31 -07:00
zinject Use abs_top_builddir when referencing libraries 2020-07-10 14:26:32 -07:00
zpool Remove dependency on sharetab file and refactor sharing logic 2020-07-13 09:19:18 -07:00
zstream Use abs_top_builddir when referencing libraries 2020-07-10 14:26:32 -07:00
zstreamdump Minor zstream redup command fixes 2020-04-10 21:10:09 -07:00
ztest Extend zdb to print inconsistencies in livelists and metaslabs 2020-07-14 17:51:05 -07:00
zvol_id OpenZFS restructuring - libspl 2019-10-02 10:39:48 -07:00
zvol_wait zvol_wait should ignore redacted zvols 2019-11-06 10:51:19 -08:00
Makefile.am Small program that converts a dataset id and an object id to a path 2020-05-20 10:05:33 -07:00