OpenZFS 9403 - assertion failed in arc_buf_destroy()

Assertion failed in arc_buf_destroy() when concurrently reading
block with checksum error.

Porting notes:
* The ability to zinject decompression errors has been added, but
  this only works at the zio_decompress() level, where we have all
  of the info we need to match against the user's zinject options.
* The decompress_fault test has been added to test the new zinject
  functionality
* We attempted to set zio_decompress_fail_fraction to (1 << 18) in
  ztest for further test coverage. Although this did uncover a few
  low priority issues, this unfortuantely also causes ztest to
  ASSERT in many locations where the code is working correctly since
  it is designed to fail on IO errors. Developers can manually set
  this variable with the '-o' option to find and debug issues.

Authored by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Matt Ahrens <mahrens@delphix.com>
Ported-by: Tom Caputi <tcaputi@datto.com>

OpenZFS-issue: https://illumos.org/issues/9403
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/fa98e487a9
Closes #7822
This commit is contained in:
Tom Caputi
2018-08-29 14:33:33 -04:00
committed by Brian Behlendorf
parent 47ab01a18f
commit c3bd3fb4ac
12 changed files with 162 additions and 39 deletions
+6 -4
View File
@@ -116,9 +116,9 @@
* specified.
*
* The '-e' option takes a string describing the errno to simulate. This must
* be one of 'io', 'checksum', or 'decrypt'. In most cases this will result
* in the same behavior, but RAID-Z will produce a different set of ereports
* for this situation.
* be one of 'io', 'checksum', 'decompress', or 'decrypt'. In most cases this
* will result in the same behavior, but RAID-Z will produce a different set of
* ereports for this situation.
*
* The '-a', '-u', and '-m' flags toggle internal flush behavior. If '-a' is
* specified, then the ARC cache is flushed appropriately. If '-u' is
@@ -300,7 +300,7 @@ usage(void)
"\n"
"\t\t-q\tQuiet mode. Only print out the handler number added.\n"
"\t\t-e\tInject a specific error. Must be one of 'io',\n"
"\t\t\t'checksum', or decrypt. Default is 'io'.\n"
"\t\t\t'checksum', 'decompress', or decrypt. Default is 'io'.\n"
"\t\t-l\tInject error at a particular block level. Default is "
"0.\n"
"\t\t-m\tAutomatically remount underlying filesystem.\n"
@@ -774,6 +774,8 @@ main(int argc, char **argv)
error = EIO;
} else if (strcasecmp(optarg, "checksum") == 0) {
error = ECKSUM;
} else if (strcasecmp(optarg, "decompress") == 0) {
error = EINVAL;
} else if (strcasecmp(optarg, "decrypt") == 0) {
error = EACCES;
} else if (strcasecmp(optarg, "nxio") == 0) {
+2 -4
View File
@@ -193,7 +193,7 @@ static const ztest_shared_opts_t ztest_opts_defaults = {
.zo_init = 1,
.zo_time = 300, /* 5 minutes */
.zo_maxloops = 50, /* max loops during spa_freeze() */
.zo_metaslab_force_ganging = 32 << 10
.zo_metaslab_force_ganging = 32 << 10,
};
extern uint64_t metaslab_force_ganging;
@@ -204,6 +204,7 @@ extern boolean_t zfs_compressed_arc_enabled;
extern int zfs_abd_scatter_enabled;
extern int dmu_object_alloc_chunk_shift;
extern boolean_t zfs_force_some_double_word_sm_entries;
extern unsigned long zio_decompress_fail_fraction;
static ztest_shared_opts_t *ztest_shared_opts;
static ztest_shared_opts_t ztest_opts;
@@ -483,9 +484,6 @@ static int zc_cb_counter = 0;
*/
#define ZTEST_COMMIT_CB_THRESH (TXG_CONCURRENT_STATES + 1000)
extern uint64_t metaslab_gang_bang;
extern uint64_t metaslab_df_alloc_threshold;
enum ztest_object {
ZTEST_META_DNODE = 0,
ZTEST_DIROBJ,