Go to file
Matthew Ahrens 6774931dfa
Extend zdb to print inconsistencies in livelists and metaslabs
Livelists and spacemaps are data structures that are logs of allocations
and frees.  Livelists entries are block pointers (blkptr_t). Spacemaps
entries are ranges of numbers, most often used as to track
allocated/freed regions of metaslabs/vdevs.

These data structures can become self-inconsistent, for example if a
block or range can be "double allocated" (two allocation records without
an intervening free) or "double freed" (two free records without an
intervening allocation).

ZDB (as well as zfs running in the kernel) can detect these
inconsistencies when loading livelists and metaslab.  However, it
generally halts processing when the error is detected.

When analyzing an on-disk problem, we often want to know the entire set
of inconsistencies, which is not possible with the current behavior.
This commit adds a new flag, `zdb -y`, which analyzes the livelist and
metaslab data structures and displays all of their inconsistencies.
Note that this is different from the leak detection performed by
`zdb -b`, which checks for inconsistencies between the spacemaps and the
tree of block pointers, but assumes the spacemaps are self-consistent.

The specific checks added are:

Verify livelists by iterating through each sublivelists and:
- report leftover FREEs
- report double ALLOCs and double FREEs
- record leftover ALLOCs together with their TXG [see Cross Check]

Verify spacemaps by iterating over each metaslab and:
- iterate over spacemap and then the metaslab's entries in the
  spacemap log, then report any double FREEs and double ALLOCs

Verify that livelists are consistenet with spacemaps.  The space
referenced by livelists (after using the FREE's to cancel out
corresponding ALLOCs) should be allocated, according to the spacemaps.

Reviewed-by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Sara Hartse <sara.hartse@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
External-issue: DLPX-66031
Closes #10515
2020-07-14 17:51:05 -07:00
.github Move cppcheck suppressions out of .github 2020-06-27 17:40:15 -07:00
cmd Extend zdb to print inconsistencies in livelists and metaslabs 2020-07-14 17:51:05 -07:00
config Centralize variable substitution 2020-07-14 17:33:44 -07:00
contrib Centralize variable substitution 2020-07-14 17:33:44 -07:00
etc Centralize variable substitution 2020-07-14 17:33:44 -07:00
include Extend zdb to print inconsistencies in livelists and metaslabs 2020-07-14 17:51:05 -07:00
lib Disable -Wl,-z,defs for ASAN builds 2020-07-14 12:17:44 -07:00
man Extend zdb to print inconsistencies in livelists and metaslabs 2020-07-14 17:51:05 -07:00
module Extend zdb to print inconsistencies in livelists and metaslabs 2020-07-14 17:51:05 -07:00
rpm Clean up lib dependencies 2020-07-10 14:26:00 -07:00
scripts Centralize variable substitution 2020-07-14 17:33:44 -07:00
tests Extend zdb to print inconsistencies in livelists and metaslabs 2020-07-14 17:51:05 -07:00
udev Centralize variable substitution 2020-07-14 17:33:44 -07:00
.editorconfig Add an .editorconfig; document git whitespace settings 2020-01-27 13:32:52 -08:00
.gitignore Add FreeBSD support to OpenZFS 2020-04-14 11:36:28 -07:00
.gitmodules Add zimport.sh compatibility test script 2014-02-21 12:10:31 -08:00
.travis.yml Add .travis.yml 2017-11-13 09:18:18 -08:00
AUTHORS Update build system and packaging 2018-05-29 16:00:33 -07:00
autogen.sh Cause autogen.sh to fail if autoreconf fails 2018-07-06 09:27:37 -07:00
CODE_OF_CONDUCT.md Add CODE_OF_CONDUCT.md 2019-04-30 10:58:45 -07:00
configure.ac Clean up lib dependencies 2020-07-10 14:26:00 -07:00
copy-builtin Move zfs_gitrev.h to build directory 2020-06-24 18:19:28 -07:00
COPYRIGHT Fix typos 2020-06-09 21:24:09 -07:00
cppcheck-suppressions.txt Move cppcheck suppressions out of .github 2020-06-27 17:40:15 -07:00
LICENSE Update build system and packaging 2018-05-29 16:00:33 -07:00
Makefile.am Centralize variable substitution 2020-07-14 17:33:44 -07:00
META Linux 5.7 compat: blk_alloc_queue() 2020-04-09 09:16:46 -07:00
NEWS Add NEWS file 2018-09-18 12:03:47 -07:00
NOTICE Update build system and packaging 2018-05-29 16:00:33 -07:00
README.md Update wiki links with new address 2020-06-03 19:46:31 -07:00
TEST Remove CI builder customization from TEST 2020-03-16 10:46:03 -07:00
zfs.release.in Move zfs.release generation to configure step 2012-07-12 12:22:51 -07:00

img

OpenZFS is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS community. This repository contains the code for running OpenZFS on Linux and FreeBSD.

codecov coverity

Official Resources

Installation

Full documentation for installing OpenZFS on your favorite Linux distribution can be found at the ZoL Site.

Contribute & Develop

We have a separate document with contribution guidelines.

We have a Code of Conduct.

Release

OpenZFS is released under a CDDL license. For more details see the NOTICE, LICENSE and COPYRIGHT files; UCRL-CODE-235197

Supported Kernels

  • The META file contains the officially recognized supported Linux kernel versions.
  • Supported FreeBSD versions are 12-STABLE and 13-CURRENT.