mirror_zfs/module/zfs
George Amanakis 657fd33bcf
Improvements on persistent L2ARC
Functional changes:

We implement refcounts of log blocks and their aligned size on the
cache device along with two corresponding arcstats. The refcounts are
reflected in the header of the device and provide valuable information
as to whether log blocks are accounted for correctly. These are
dynamically adjusted as log blocks are committed/evicted. zdb also uses
this information in the device header and compares it to the
corresponding values as reported by dump_l2arc_log_blocks() which
emulates l2arc_rebuild(). If the refcounts saved in the device header
report higher values, zdb exits with an error. For this feature to work
correctly there should be no active writes on the device. This is also
employed in the tests of persistent L2ARC. We extend the structure of
the cache device header by adding the two new variables mirroring the
refcounts after the existing variables to preserve backward
compatibility in terms of persistent L2ARC.

1) a new arcstat "l2_log_blk_asize" and refcount "l2ad_lb_asize" which
   reflect the total aligned size of log blocks on the device. This is
   also reflected in the header of the cache device as "dh_lb_asize".
2) a new arcstat "l2arc_log_blk_count" and refcount "l2ad_lb_count"
   which reflect the total number of L2ARC log blocks present on cache
   devices.  It is also reflected in the header of the cache device as
   "dh_lb_count".

In l2arc_rebuild_vdev() if the amount of committed log entries in a log
block is 0 and the device header is valid we update the device header.
This will facilitate trimming of the whole device in this case when
TRIM for L2ARC is implemented.

Improve loop protection in l2arc_rebuild() by using the starting offset
of the payload of each log block instead of the starting offset of the
log block.

If the zio in l2arc_write_buffers() fails, restore the lbps array in the
header of the device to its previous state in l2arc_write_done().

If l2arc_rebuild() ends the rebuild process without restoring any L2ARC
log blocks in ARC and without any other error, this means that the lbps
array in the header is pointing to non-existent or invalid log blocks.
Reset the device header in this case.

In l2arc_rebuild() change the zfs_dbgmsg messages to
spa_history_log_internal() making them user visible with zpool history
command.

Non-functional changes:

Make the first test in persistent L2ARC use `zdb -lll` to increase
coverage in `zdb.c`.

Rename psize with asize when referring to log blocks, since
L2ARC_SET_PSIZE stores the vdev aligned size for log blocks. Also
rename dh_log_blk_entries to dh_log_entries to make it clear that
it is a mirror of l2ad_log_entries. Added comments for both changes.

Fix inaccurate comments for example in l2arc_log_blk_restore().

Add asserts at the end in l2arc_evict() and l2arc_write_buffers().

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: George Amanakis <gamanakis@gmail.com>
Closes #10228
2020-05-07 16:34:03 -07:00
..
aggsum.c Reduce number of atomic_add() calls in aggsum 2020-02-06 13:21:06 -08:00
arc.c Improvements on persistent L2ARC 2020-05-07 16:34:03 -07:00
blkptr.c Undo c89 workarounds to match with upstream 2017-11-04 13:25:13 -07:00
bplist.c Fast Clone Deletion 2019-07-26 10:54:14 -07:00
bpobj.c Add subcommand to wait for background zfs activity to complete 2019-09-13 18:09:06 -07:00
bptree.c Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
bqueue.c Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
btree.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
dataset_kstats.c Fix panic on DilOS with kstat per dataset statistics 2019-09-03 12:12:31 -07:00
dbuf_stats.c Make module tunables cross platform 2019-09-05 14:49:49 -07:00
dbuf.c Compile cityhash code into libzfs 2020-03-27 09:11:22 -07:00
ddt_zap.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
ddt.c Update FreeBSD tunables 2020-04-15 11:14:47 -07:00
dmu_diff.c Mark write_record static 2019-12-03 09:51:44 -08:00
dmu_object.c Make module tunables cross platform 2019-09-05 14:49:49 -07:00
dmu_objset.c dmu_objset_from_ds must be called with dp_config_rwlock held 2020-03-12 10:55:02 -07:00
dmu_recv.c Fix missing ivset guid with resumed raw base recv 2020-04-24 19:00:32 -07:00
dmu_redact.c dmu_objset_from_ds must be called with dp_config_rwlock held 2020-03-12 10:55:02 -07:00
dmu_send.c dmu_objset_from_ds must be called with dp_config_rwlock held 2020-03-12 10:55:02 -07:00
dmu_traverse.c Make module tunables cross platform 2019-09-05 14:49:49 -07:00
dmu_tx.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
dmu_zfetch.c Make module tunables cross platform 2019-09-05 14:49:49 -07:00
dmu.c Remove deduplicated send/receive code 2020-04-23 10:06:57 -07:00
dnode_sync.c Convert dbuf dirty record record list to a list_t 2020-02-05 11:07:19 -08:00
dnode.c Prevent race condition in dnode_dest (#10101) 2020-03-12 10:25:56 -07:00
dsl_bookmark.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
dsl_crypt.c Add missing zfs_refcount_destroy() in key_mapping_rele() 2020-04-28 09:53:45 -07:00
dsl_dataset.c Add 'zfs wait' command 2020-04-01 10:02:06 -07:00
dsl_deadlist.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
dsl_deleg.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
dsl_destroy.c Add 'zfs wait' command 2020-04-01 10:02:06 -07:00
dsl_dir.c Add 'zfs wait' command 2020-04-01 10:02:06 -07:00
dsl_pool.c Eliminate Linux specific inode usage from common code 2019-12-11 11:53:57 -08:00
dsl_prop.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
dsl_scan.c Fix infinite scan on a pool with only special allocations 2020-03-12 10:52:03 -07:00
dsl_synctask.c Fix typos in module/zfs/ 2019-09-02 17:56:41 -07:00
dsl_userhold.c Fix strdup conflict on other platforms 2019-10-10 09:47:06 -07:00
edonr_zfs.c DLPX-44812 integrate EP-220 large memory scalability 2016-11-29 14:34:27 -08:00
fm.c Enable zpool events tunables and tests on FreeBSD 2020-02-18 11:22:56 -08:00
gzip.c OpenZFS restructuring - move platform specific sources 2019-09-06 11:26:26 -07:00
hkdf.c Encryption patch follow-up 2017-10-11 16:54:48 -04:00
lz4.c Enable clang to use intrinsics for lz4 2019-10-01 13:17:32 -07:00
lzjb.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
Makefile.in Compile cityhash code into libzfs 2020-03-27 09:11:22 -07:00
metaslab.c Use a struct to organize metaslab-group-allocator fields 2020-04-22 10:26:56 -07:00
mmp.c Improve zfs destroy performance with zio_t-free zio_free() 2020-02-28 14:49:44 -08:00
multilist.c Enable use of DTRACE_PROBE* macros in "spl" module 2019-11-01 13:13:43 -07:00
objlist.c Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
pathname.c Disable unused pathname::pn_path* (unneeded in Linux) 2019-07-15 13:57:56 -07:00
range_tree.c Function name and comment updates 2019-10-11 10:13:21 -07:00
refcount.c Disable user space reference tracking 2020-04-13 10:51:44 -07:00
rrwlock.c Enable use of DTRACE_PROBE* macros in "spl" module 2019-11-01 13:13:43 -07:00
sa.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
sha256.c OpenZFS restructuring - move platform specific sources 2019-09-06 11:26:26 -07:00
skein_zfs.c DLPX-44812 integrate EP-220 large memory scalability 2016-11-29 14:34:27 -08:00
spa_boot.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
spa_checkpoint.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
spa_config.c Add zfs_file_* interface, remove vnodes 2019-11-21 09:32:57 -08:00
spa_errlog.c Fix typos in module/zfs/ 2019-09-02 17:56:41 -07:00
spa_history.c Make spa_history_zone platform-dependent in kernel 2020-03-02 09:43:30 -08:00
spa_log_spacemap.c Make module tunables cross platform 2019-09-05 14:49:49 -07:00
spa_misc.c Finish refactoring for ZFS_MODULE_PARAM_CALL 2020-04-07 10:06:22 -07:00
spa.c Enable splitting mirrors with indirect vdevs 2020-05-06 10:32:28 -07:00
space_map.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
space_reftree.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
THIRDPARTYLICENSE.cityhash OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
THIRDPARTYLICENSE.cityhash.descrip OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
txg.c Update FreeBSD tunables 2020-04-15 11:14:47 -07:00
uberblock.c MMP interval and fail_intervals in uberblock 2019-03-21 12:47:57 -07:00
unique.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
vdev_cache.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
vdev_indirect_births.c Fixes: #8934 Large kmem_alloc 2019-07-10 15:54:49 -07:00
vdev_indirect_mapping.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
vdev_indirect.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
vdev_initialize.c Linux 5.6 compat: time_t 2020-02-27 09:31:02 -08:00
vdev_label.c Add support for boot environment data to be stored in the label 2020-05-07 09:36:33 -07:00
vdev_mirror.c Avoid some crashes when importing a pool with corrupt metadata 2019-12-26 10:57:05 -08:00
vdev_missing.c Update vdev_ops_t from illumos 2019-06-20 18:29:02 -07:00
vdev_queue.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
vdev_raidz_math_aarch64_neon_common.h Minor performance fix for NEON RAID-Z 2019-12-17 19:34:52 -08:00
vdev_raidz_math_aarch64_neon.c Linux 5.0 compat: SIMD compatibility 2019-07-12 09:31:20 -07:00
vdev_raidz_math_aarch64_neonx2.c Linux 5.0 compat: SIMD compatibility 2019-07-12 09:31:20 -07:00
vdev_raidz_math_avx2.c OpenZFS restructuring - move platform specific headers 2019-09-05 09:34:54 -07:00
vdev_raidz_math_avx512bw.c OpenZFS restructuring - move platform specific headers 2019-09-05 09:34:54 -07:00
vdev_raidz_math_avx512f.c Make clang happy with vdev_raidz_ code 2019-10-10 09:45:37 -07:00
vdev_raidz_math_impl.h Fix const-correctness in raidz math 2020-02-03 10:52:41 -08:00
vdev_raidz_math_powerpc_altivec_common.h Add AltiVec RAID-Z 2020-01-23 11:01:24 -08:00
vdev_raidz_math_powerpc_altivec.c Add AltiVec RAID-Z 2020-01-23 11:01:24 -08:00
vdev_raidz_math_scalar.c Linux 5.3: Fix switch() fall though compiler errors 2019-08-21 09:29:23 -07:00
vdev_raidz_math_sse2.c Make clang happy with vdev_raidz_ code 2019-10-10 09:45:37 -07:00
vdev_raidz_math_ssse3.c OpenZFS restructuring - move platform specific headers 2019-09-05 09:34:54 -07:00
vdev_raidz_math.c Add AltiVec RAID-Z 2020-01-23 11:01:24 -08:00
vdev_raidz.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
vdev_removal.c Cancel initialize and TRIM before vdev_metaslab_fini() 2019-12-26 10:50:23 -08:00
vdev_root.c Enable splitting mirrors with indirect vdevs 2020-05-06 10:32:28 -07:00
vdev_trim.c Add trim support to zpool wait 2020-03-04 15:07:11 -08:00
vdev.c Add support for boot environment data to be stored in the label 2020-05-07 09:36:33 -07:00
zap_leaf.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
zap_micro.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
zap.c Make module tunables cross platform 2019-09-05 14:49:49 -07:00
zcp_get.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
zcp_global.c OpenZFS 8600 - ZFS channel programs - snapshot 2018-02-08 15:29:24 -08:00
zcp_iter.c Fix typos in module/zfs/ 2019-09-02 17:56:41 -07:00
zcp_set.c Support setting user properties in a channel program 2020-02-14 13:41:42 -08:00
zcp_synctask.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
zcp.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
zfeature.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
zfs_byteswap.c Fix typos in module/zfs/ 2019-09-02 17:56:41 -07:00
zfs_fm.c Add zpool status -s (slow I/Os) and -p (parseable) 2018-11-08 16:47:24 -08:00
zfs_fuid.c Relocate common quota functions to shared code 2019-12-11 12:12:08 -08:00
zfs_ioctl.c Add support for boot environment data to be stored in the label 2020-05-07 09:36:33 -07:00
zfs_log.c Don't call zrele on passed zp in zfs_xattr_owner_unlinked on FreeBSD 2020-02-28 14:53:18 -08:00
zfs_onexit.c Remove deduplicated send/receive code 2020-04-23 10:06:57 -07:00
zfs_quota.c Relocate common quota functions to shared code 2019-12-11 12:12:08 -08:00
zfs_ratelimit.c Change checksum & IO delay ratelimit values 2018-03-04 17:34:51 -08:00
zfs_replay.c Simplify FreeBSD's locking requirements in zfs_replay.c 2020-01-22 17:55:56 -08:00
zfs_rlock.c Move platform specific parts of zfs_znode.h to platform code 2019-11-06 10:54:25 -08:00
zfs_sa.c Add inode accessors to common code 2019-10-02 09:15:12 -07:00
zil.c Improve logging of 128KB writes 2019-11-11 09:27:59 -08:00
zio_checksum.c Disable EDONR on FreeBSD 2019-12-05 13:10:29 -08:00
zio_compress.c zio_decompress_data always ASSERTs successful decompression 2019-12-10 15:51:58 -08:00
zio_inject.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
zio.c Compile cityhash code into libzfs 2020-03-27 09:11:22 -07:00
zle.c Fix zle_decompress out of bound access 2018-02-09 10:08:05 -08:00
zrlock.c Enable use of DTRACE_PROBE* macros in "spl" module 2019-11-01 13:13:43 -07:00
zthr.c Fast Clone Deletion 2019-07-26 10:54:14 -07:00
zvol.c async zvol minor node creation interferes with receive 2020-02-03 09:33:14 -08:00