Matthew Ahrens 6774931dfa Extend zdb to print inconsistencies in livelists and metaslabs ... Livelists and spacemaps are data structures that are logs of allocations and frees. Livelists entries are block pointers (blkptr_t). Spacemaps entries are ranges of numbers, most often used as to track allocated/freed regions of metaslabs/vdevs. These data structures can become self-inconsistent, for example if a block or range can be "double allocated" (two allocation records without an intervening free) or "double freed" (two free records without an intervening allocation). ZDB (as well as zfs running in the kernel) can detect these inconsistencies when loading livelists and metaslab. However, it generally halts processing when the error is detected. When analyzing an on-disk problem, we often want to know the entire set of inconsistencies, which is not possible with the current behavior. This commit adds a new flag, `zdb -y`, which analyzes the livelist and metaslab data structures and displays all of their inconsistencies. Note that this is different from the leak detection performed by `zdb -b`, which checks for inconsistencies between the spacemaps and the tree of block pointers, but assumes the spacemaps are self-consistent. The specific checks added are: Verify livelists by iterating through each sublivelists and: - report leftover FREEs - report double ALLOCs and double FREEs - record leftover ALLOCs together with their TXG [see Cross Check] Verify spacemaps by iterating over each metaslab and: - iterate over spacemap and then the metaslab's entries in the spacemap log, then report any double FREEs and double ALLOCs Verify that livelists are consistenet with spacemaps. The space referenced by livelists (after using the FREE's to cancel out corresponding ALLOCs) should be allocated, according to the spacemaps. Reviewed-by: Serapheim Dimitropoulos <serapheim@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sara Hartse <sara.hartse@delphix.com> Signed-off-by: Matthew Ahrens <mahrens@delphix.com> External-issue: DLPX-66031 Closes #10515		2020-07-14 17:51:05 -07:00
..
abd.c	Fixing gang ABD child removal race condition	2020-07-14 11:04:35 -07:00
aggsum.c	Reduce number of atomic_add() calls in aggsum	2020-02-06 13:21:06 -08:00
arc.c	Fix a persistent L2ARC bug in l2arc_write_done()	2020-07-10 14:10:03 -07:00
blkptr.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
bplist.c	Fast Clone Deletion	2019-07-26 10:54:14 -07:00
bpobj.c	Add subcommand to wait for background zfs activity to complete	2019-09-13 18:09:06 -07:00
bptree.c	Implement Redacted Send/Receive	2019-06-19 09:48:12 -07:00
bqueue.c	Implement Redacted Send/Receive	2019-06-19 09:48:12 -07:00
btree.c	Fix typos	2020-06-09 21:24:09 -07:00
dataset_kstats.c	Fix panic on DilOS with kstat per dataset statistics	2019-09-03 12:12:31 -07:00
dbuf_stats.c	Mark functions as static	2020-06-18 12:20:38 -07:00
dbuf.c	Mark functions as static	2020-06-18 12:20:38 -07:00
ddt_zap.c	Refactor dnode dirty context from dbuf_dirty	2020-02-26 16:09:17 -08:00
ddt.c	Remove dead code	2020-06-18 12:21:18 -07:00
dmu_diff.c	Mark write_record static	2019-12-03 09:51:44 -08:00
dmu_object.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
dmu_objset.c	filesystem_limit/snapshot_limit is incorrectly enforced against root	2020-07-11 17:18:02 -07:00
dmu_recv.c	filesystem_limit/snapshot_limit is incorrectly enforced against root	2020-07-11 17:18:02 -07:00
dmu_redact.c	dmu_objset_from_ds must be called with dp_config_rwlock held	2020-03-12 10:55:02 -07:00
dmu_send.c	Replace sprintf()->snprintf() and strcpy()->strlcpy()	2020-06-07 11:42:12 -07:00
dmu_traverse.c	Make module tunables cross platform	2019-09-05 14:49:49 -07:00
dmu_tx.c	Refactor dnode dirty context from dbuf_dirty	2020-02-26 16:09:17 -08:00
dmu_zfetch.c	Make module tunables cross platform	2019-09-05 14:49:49 -07:00
dmu.c	Mark functions as static	2020-06-18 12:20:38 -07:00
dnode_sync.c	Convert dbuf dirty record record list to a list_t	2020-02-05 11:07:19 -08:00
dnode.c	Prevent race condition in dnode_dest (#10101)	2020-03-12 10:25:56 -07:00
dsl_bookmark.c	Fix typos	2020-06-09 21:24:09 -07:00
dsl_crypt.c	Mark functions as static	2020-06-18 12:20:38 -07:00
dsl_dataset.c	filesystem_limit/snapshot_limit is incorrectly enforced against root	2020-07-11 17:18:02 -07:00
dsl_deadlist.c	Reduce loaded range tree memory usage	2019-10-09 10:36:03 -07:00
dsl_deleg.c	Reduce loaded range tree memory usage	2019-10-09 10:36:03 -07:00
dsl_destroy.c	Memory leak in dsl_destroy_snapshots_nvl error case	2020-05-26 16:13:41 -07:00
dsl_dir.c	filesystem_limit/snapshot_limit is incorrectly enforced against root	2020-07-11 17:18:02 -07:00
dsl_pool.c	Mark functions as static	2020-06-18 12:20:38 -07:00
dsl_prop.c	Replace sprintf()->snprintf() and strcpy()->strlcpy()	2020-06-07 11:42:12 -07:00
dsl_scan.c	Add device rebuild feature	2020-07-03 11:05:50 -07:00
dsl_synctask.c	Fix typos in module/zfs/	2019-09-02 17:56:41 -07:00
dsl_userhold.c	Replace sprintf()->snprintf() and strcpy()->strlcpy()	2020-06-07 11:42:12 -07:00
edonr_zfs.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
fm.c	Enable zpool events tunables and tests on FreeBSD	2020-02-18 11:22:56 -08:00
gzip.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
hkdf.c
lz4.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
lzjb.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
Makefile.in	Add device rebuild feature	2020-07-03 11:05:50 -07:00
metaslab.c	Extend zdb to print inconsistencies in livelists and metaslabs	2020-07-14 17:51:05 -07:00
mmp.c	Add zfs_multihost_interval tunable handler for FreeBSD	2020-06-23 13:32:42 -07:00
multilist.c	Enable use of DTRACE_PROBE* macros in "spl" module	2019-11-01 13:13:43 -07:00
objlist.c	Implement Redacted Send/Receive	2019-06-19 09:48:12 -07:00
pathname.c	Disable unused pathname::pn_path* (unneeded in Linux)	2019-07-15 13:57:56 -07:00
range_tree.c	Function name and comment updates	2019-10-11 10:13:21 -07:00
refcount.c	Disable user space reference tracking	2020-04-13 10:51:44 -07:00
rrwlock.c	Enable use of DTRACE_PROBE* macros in "spl" module	2019-11-01 13:13:43 -07:00
sa.c	Mark functions as static	2020-06-18 12:20:38 -07:00
sha256.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
skein_zfs.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
spa_boot.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
spa_checkpoint.c	Refactor dnode dirty context from dbuf_dirty	2020-02-26 16:09:17 -08:00
spa_config.c	freebsd: changes necessary to coexist with dtrace in tree	2020-07-01 09:10:08 -07:00
spa_errlog.c	Fix typos in module/zfs/	2019-09-02 17:56:41 -07:00
spa_history.c	Make spa_history_zone platform-dependent in kernel	2020-03-02 09:43:30 -08:00
spa_log_spacemap.c	Make module tunables cross platform	2019-09-05 14:49:49 -07:00
spa_misc.c	Add device rebuild feature	2020-07-03 11:05:50 -07:00
spa.c	Fix LOR between dp_config_rwlock and spa_props_lock	2020-07-14 12:21:57 -07:00
space_map.c	Extend zdb to print inconsistencies in livelists and metaslabs	2020-07-14 17:51:05 -07:00
space_reftree.c	Reduce loaded range tree memory usage	2019-10-09 10:36:03 -07:00
THIRDPARTYLICENSE.cityhash	OpenZFS 8484 - Implement aggregate sum and use for arc counters	2018-06-06 09:35:59 -07:00
THIRDPARTYLICENSE.cityhash.descrip	OpenZFS 8484 - Implement aggregate sum and use for arc counters	2018-06-06 09:35:59 -07:00
txg.c	Use boot_ncpus in place of max_ncpus in taskq_create	2020-05-20 10:07:21 -07:00
uberblock.c	MMP interval and fail_intervals in uberblock	2019-03-21 12:47:57 -07:00
unique.c	Reduce loaded range tree memory usage	2019-10-09 10:36:03 -07:00
vdev_cache.c	Replace ASSERTV macro with compiler annotation	2019-12-05 12:37:00 -08:00
vdev_indirect_births.c	Fixes: #8934 Large kmem_alloc	2019-07-10 15:54:49 -07:00
vdev_indirect_mapping.c	Replace ASSERTV macro with compiler annotation	2019-12-05 12:37:00 -08:00
vdev_indirect.c	Mark functions as static	2020-06-18 12:20:38 -07:00
vdev_initialize.c	Upstream: add missing thread_exit()	2020-05-14 15:58:09 -07:00
vdev_label.c	Add device rebuild feature	2020-07-03 11:05:50 -07:00
vdev_mirror.c	Add device rebuild feature	2020-07-03 11:05:50 -07:00
vdev_missing.c	Update vdev_ops_t from illumos	2019-06-20 18:29:02 -07:00
vdev_queue.c	Add device rebuild feature	2020-07-03 11:05:50 -07:00
vdev_raidz_math_aarch64_neon_common.h	Minor performance fix for NEON RAID-Z	2019-12-17 19:34:52 -08:00
vdev_raidz_math_aarch64_neon.c	Linux 5.0 compat: SIMD compatibility	2019-07-12 09:31:20 -07:00
vdev_raidz_math_aarch64_neonx2.c	Linux 5.0 compat: SIMD compatibility	2019-07-12 09:31:20 -07:00
vdev_raidz_math_avx2.c	OpenZFS restructuring - move platform specific headers	2019-09-05 09:34:54 -07:00
vdev_raidz_math_avx512bw.c	OpenZFS restructuring - move platform specific headers	2019-09-05 09:34:54 -07:00
vdev_raidz_math_avx512f.c	Make clang happy with vdev_raidz_ code	2019-10-10 09:45:37 -07:00
vdev_raidz_math_impl.h	Fix const-correctness in raidz math	2020-02-03 10:52:41 -08:00
vdev_raidz_math_powerpc_altivec_common.h	Add AltiVec RAID-Z	2020-01-23 11:01:24 -08:00
vdev_raidz_math_powerpc_altivec.c	Fix typos	2020-06-09 21:24:09 -07:00
vdev_raidz_math_scalar.c	Linux 5.3: Fix switch() fall though compiler errors	2019-08-21 09:29:23 -07:00
vdev_raidz_math_sse2.c	Make clang happy with vdev_raidz_ code	2019-10-10 09:45:37 -07:00
vdev_raidz_math_ssse3.c	OpenZFS restructuring - move platform specific headers	2019-09-05 09:34:54 -07:00
vdev_raidz_math.c	Add prototypes	2020-06-18 12:21:32 -07:00
vdev_raidz.c	Fix typos	2020-06-09 21:24:09 -07:00
vdev_rebuild.c	Add device rebuild feature	2020-07-03 11:05:50 -07:00
vdev_removal.c	Trim L2ARC	2020-06-09 10:15:08 -07:00
vdev_root.c	Enable splitting mirrors with indirect vdevs	2020-05-06 10:32:28 -07:00
vdev_trim.c	Trim L2ARC	2020-06-09 10:15:08 -07:00
vdev.c	Add device rebuild feature	2020-07-03 11:05:50 -07:00
zap_leaf.c	Refactor dnode dirty context from dbuf_dirty	2020-02-26 16:09:17 -08:00
zap_micro.c	Mark functions as static	2020-06-18 12:20:38 -07:00
zap.c	Replace sprintf()->snprintf() and strcpy()->strlcpy()	2020-06-07 11:42:12 -07:00
zcp_get.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
zcp_global.c	OpenZFS 8600 - ZFS channel programs - snapshot	2018-02-08 15:29:24 -08:00
zcp_iter.c	Fix typos in module/zfs/	2019-09-02 17:56:41 -07:00
zcp_set.c	Support setting user properties in a channel program	2020-02-14 13:41:42 -08:00
zcp_synctask.c	filesystem_limit/snapshot_limit is incorrectly enforced against root	2020-07-11 17:18:02 -07:00
zcp.c	filesystem_limit/snapshot_limit is incorrectly enforced against root	2020-07-11 17:18:02 -07:00
zfeature.c	Replace ASSERTV macro with compiler annotation	2019-12-05 12:37:00 -08:00
zfs_byteswap.c	Mark functions as static	2020-06-18 12:20:38 -07:00
zfs_fm.c	Add zpool status -s (slow I/Os) and -p (parseable)	2018-11-08 16:47:24 -08:00
zfs_fuid.c	Replace sprintf()->snprintf() and strcpy()->strlcpy()	2020-06-07 11:42:12 -07:00
zfs_ioctl.c	Add device rebuild feature	2020-07-03 11:05:50 -07:00
zfs_log.c	Add prototypes	2020-06-18 12:21:32 -07:00
zfs_onexit.c	Remove deduplicated send/receive code	2020-04-23 10:06:57 -07:00
zfs_quota.c	File incorrectly zeroed when receiving incremental stream that toggles -L	2020-06-09 10:41:01 -07:00
zfs_ratelimit.c	Change checksum & IO delay ratelimit values	2018-03-04 17:34:51 -08:00
zfs_replay.c	Simplify FreeBSD's locking requirements in zfs_replay.c	2020-01-22 17:55:56 -08:00
zfs_rlock.c	Add a "try" operation for range locks	2020-07-06 11:53:31 -07:00
zfs_sa.c	Add convenience wrappers for common uio usage	2020-06-14 10:09:55 -07:00
zil.c	Mark functions as static	2020-06-18 12:20:38 -07:00
zio_checksum.c	Mark functions as static	2020-06-18 12:20:38 -07:00
zio_compress.c	zio_decompress_data always ASSERTs successful decompression	2019-12-10 15:51:58 -08:00
zio_inject.c	Replace ASSERTV macro with compiler annotation	2019-12-05 12:37:00 -08:00
zio.c	Mark functions as static	2020-06-18 12:20:38 -07:00
zle.c	Add include files for prototypes	2020-06-18 12:21:25 -07:00
zrlock.c	Remove dead code	2020-06-18 12:21:18 -07:00
zthr.c	Fast Clone Deletion	2019-07-26 10:54:14 -07:00
zvol.c	Fix typos	2020-06-09 21:24:09 -07:00