Alexander Motin a8d83e2a24 More adaptive ARC eviction ... Traditionally ARC adaptation was limited to MRU/MFU distribution. But for years people with metadata-centric workload demanded mechanisms to also manage data/metadata distribution, that in original ZFS was just a FIFO. As result ZFS effectively got separate states for data and metadata, minimum and maximum metadata limits etc, but it all required manual tuning, was not adaptive and in its heart remained a bad FIFO. This change removes most of existing eviction logic, rewriting it from scratch. This makes MRU/MFU adaptation individual for data and meta- data, same as the distribution between data and metadata themselves. Since most of required states separation was already done, it only required to make arcs_size state field specific per data/metadata. The adaptation logic is still based on previous concept of ghost hits, just now it balances ARC capacity between 4 states: MRU data, MRU metadata, MFU data and MFU metadata. To simplify arc_c changes instead of arc_p measured in bytes, this code uses 3 variable arc_meta, arc_pd and arc_pm, representing ARC balance between metadata and data, MRU and MFU for data, and MRU and MFU for metadata respectively as 32-bit fixed point fractions. Since we care about the math result only when need to evict, this moves all the logic from arc_adapt() to arc_evict(), that reduces per-block overhead, since per-block operations are limited to stats collection, now moved from arc_adapt() to arc_access() and using cheaper wmsums. This also allows to remove ugly ARC_HDR_DO_ADAPT flag from many places. This change also removes number of metadata specific tunables, part of which were actually not functioning correctly, since not all metadata are equal and some (like L2ARC headers) are not really evictable. Instead it introduced single opaque knob zfs_arc_meta_balance, tuning ARC's reaction on ghost hits, allowing administrator give more or less preference to metadata without setting strict limits. Some of old code parts like arc_evict_meta() are just removed, because since introduction of ABD ARC they really make no sense: only headers referenced by small number of buffers are not evictable, and they are really not evictable no matter what this code do. Instead just call arc_prune_async() if too much metadata appear not evictable. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Allan Jude <allan@klarasystems.com> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored by: iXsystems, Inc. Closes #14359		2023-03-08 11:17:23 -08:00
..
abd.c	abd_return_buf() should call zfs_refcount_remove_many() early	2022-10-19 17:11:01 -07:00
aggsum.c	Remove bcopy(), bzero(), bcmp()	2022-03-15 15:13:42 -07:00
arc.c	More adaptive ARC eviction	2023-03-08 11:17:23 -08:00
blake3_zfs.c	Cleanup: Use NULL when doing NULL pointer comparisons	2023-01-12 16:00:37 -08:00
blkptr.c	Remove bcopy(), bzero(), bcmp()	2022-03-15 15:13:42 -07:00
bplist.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
bpobj.c	Prefetch on deadlists merge	2023-01-25 11:30:24 -08:00
bptree.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
bqueue.c	Batch enqueue/dequeue for bqueue	2023-01-10 13:39:22 -08:00
btree.c	Optimize microzaps	2022-10-20 11:57:15 -07:00
dataset_kstats.c	Introduce kmem_scnprintf()	2022-10-29 13:05:11 -07:00
dbuf_stats.c	Revert "Reduce dbuf_find() lock contention"	2022-09-22 12:59:41 -07:00
dbuf.c	Optimize the is_l2cacheable functions	2023-03-06 16:13:05 -08:00
ddt_zap.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
ddt.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
dmu_diff.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
dmu_object.c	zfs redact fails when dnodesize=auto	2023-02-16 09:23:39 -08:00
dmu_objset.c	Optimize the is_l2cacheable functions	2023-03-06 16:13:05 -08:00
dmu_recv.c	Skip memory allocation when compressing holes	2023-02-27 14:41:02 -08:00
dmu_redact.c	Fix incorrect size given to bqueue_enqueue() call in dmu_redact.c	2022-09-15 16:21:21 -07:00
dmu_send.c	Give strlcat() full buffer lengths rather than smaller buffer lengths	2023-02-14 11:03:42 -08:00
dmu_traverse.c	arc_read()/arc_access() refactoring and cleanup	2022-12-22 12:10:24 -08:00
dmu_tx.c	Add tunable to allow changing micro ZAP's max size	2023-01-10 13:41:54 -08:00
dmu_zfetch.c	System-wide speculative prefetch limit.	2023-03-01 15:27:40 -08:00
dmu.c	Use rw_tryupgrade() in dmu_bonus_hold_by_dnode()	2023-02-22 16:33:23 -08:00
dnode_sync.c	free_blocks(): Fix reports from 2016 PVS Studio FreeBSD report	2023-01-23 13:12:37 -08:00
dnode.c	Turn default_bs and default_ibs into ZFS_MODULE_PARAMs	2023-01-11 09:38:20 -08:00
dsl_bookmark.c	Cleanup: Address Clang's static analyzer's unused code complaints	2022-10-14 13:37:54 -07:00
dsl_crypt.c	Better handling for future crypto parameters	2023-03-07 14:05:14 -08:00
dsl_dataset.c	Revert zfeature_active() to static	2023-02-28 14:03:52 -08:00
dsl_deadlist.c	Add missing increment to dsl_deadlist_move_bpobj()	2023-03-06 15:28:26 -08:00
dsl_deleg.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
dsl_destroy.c	Prevent zevent list from consuming all of kernel memory	2022-08-22 12:36:22 -07:00
dsl_dir.c	Cleanup ->dd_space_towrite should be unsigned	2023-01-20 11:10:15 -08:00
dsl_pool.c	Move dmu_buf_rele() after dsl_dataset_sync_done()	2023-02-23 18:14:52 -07:00
dsl_prop.c	Micro-optimize dsl_prop_get_dd()	2023-01-20 11:01:41 -08:00
dsl_scan.c	Partially revert eee9362a7	2023-02-21 09:36:22 -08:00
dsl_synctask.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
dsl_userhold.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
edonr_zfs.c	Remove bcopy(), bzero(), bcmp()	2022-03-15 15:13:42 -07:00
fm.c	Cleanup of dead code suggested by Clang Static Analyzer (#14380)	2023-01-17 09:57:12 -08:00
gzip.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
hkdf.c	Remove bcopy(), bzero(), bcmp()	2022-03-15 15:13:42 -07:00
lz4_zfs.c	Updated the lz4 decompressor	2022-01-07 10:36:49 -08:00
lz4.c	lz4: Cherrypick fix for CVE-2021-3520	2022-01-12 16:14:36 -08:00
lzjb.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
metaslab.c	Bypass metaslab throttle for removal allocations	2022-12-09 10:48:33 -08:00
mmp.c	Cleanup: Address Clang's static analyzer's unused code complaints	2022-10-14 13:37:54 -07:00
multilist.c	Cleanup: Specify unsignedness on things that should not be signed	2022-09-27 16:42:41 -07:00
objlist.c	Implement Redacted Send/Receive	2019-06-19 09:48:12 -07:00
pathname.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
range_tree.c	Add defensive assertions	2022-10-12 11:25:18 -07:00
refcount.c	Cleanup: Specify unsignedness on things that should not be signed	2022-09-27 16:42:41 -07:00
rrwlock.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
sa.c	Suppress static analyzer warning in sa_attr_iter()	2023-02-28 17:31:30 -08:00
sha2_zfs.c	Add generic implementation handling and SHA2 impl	2023-03-02 13:52:21 -08:00
skein_zfs.c	Remove bcopy(), bzero(), bcmp()	2022-03-15 15:13:42 -07:00
spa_checkpoint.c	Cleanup: 64-bit kernel module parameters should use fixed width types	2022-10-13 10:03:29 -07:00
spa_config.c	zed: add hotplug support for spare vdevs	2023-01-09 12:43:03 -08:00
spa_errlog.c	deadlock between spa_errlog_lock and dp_config_rwlock	2022-12-22 11:48:49 -08:00
spa_history.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
spa_log_spacemap.c	Address warnings about possible division by zero from clangsa	2022-11-03 09:58:14 -07:00
spa_misc.c	Improve resilver ETAs	2023-01-25 11:28:54 -08:00
spa_stats.c	Cleanup: Specify unsignedness on things that should not be signed	2022-09-27 16:42:41 -07:00
spa.c	Sync thread should avoid holding the spa config write lock when possible	2023-02-16 14:10:52 -08:00
space_map.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
space_reftree.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
THIRDPARTYLICENSE.cityhash	OpenZFS 8484 - Implement aggregate sum and use for arc counters	2018-06-06 09:35:59 -07:00
THIRDPARTYLICENSE.cityhash.descrip	OpenZFS 8484 - Implement aggregate sum and use for arc counters	2018-06-06 09:35:59 -07:00
txg.c	Fix the last two CFI callback prototype mismatches	2022-11-29 09:56:16 -08:00
uberblock.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
unique.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_cache.c	Cleanup: Specify unsignedness on things that should not be signed	2022-09-27 16:42:41 -07:00
vdev_draid_rand.c	Distributed Spare (dRAID) Feature	2020-11-13 13:51:51 -08:00
vdev_draid.c	vdev_draid_lookup_map() should not iterate outside draid_maps	2022-09-12 12:51:17 -07:00
vdev_indirect_births.c	Remove bcopy(), bzero(), bcmp()	2022-03-15 15:13:42 -07:00
vdev_indirect_mapping.c	Remove bcopy(), bzero(), bcmp()	2022-03-15 15:13:42 -07:00
vdev_indirect.c	Cleanup: Replace oldstyle struct hack with C99 flexible array members	2023-01-12 16:00:03 -08:00
vdev_initialize.c	Cleanup: 64-bit kernel module parameters should use fixed width types	2022-10-13 10:03:29 -07:00
vdev_label.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_mirror.c	Improve too large physical ashift handling	2022-09-08 10:30:53 -07:00
vdev_missing.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_queue.c	Convert enum zio_flag to uint64_t	2022-10-27 09:54:54 -07:00
vdev_raidz_math_aarch64_neon_common.h	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_raidz_math_aarch64_neon.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_raidz_math_aarch64_neonx2.c	Fix Clang 15 compilation errors	2022-11-30 13:46:26 -08:00
vdev_raidz_math_avx2.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_raidz_math_avx512bw.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_raidz_math_avx512f.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_raidz_math_impl.h	Cleanup Raid-Z Typo fixes	2022-09-06 09:43:21 -07:00
vdev_raidz_math_powerpc_altivec_common.h	Linux ppc64le ieee128 compat: Do not redefine __asm on external headers	2023-01-13 10:58:58 -08:00
vdev_raidz_math_powerpc_altivec.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_raidz_math_scalar.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_raidz_math_sse2.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_raidz_math_ssse3.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_raidz_math.c	Convert some sprintf() calls to kmem_scnprintf()	2022-11-28 13:49:58 -08:00
vdev_raidz.c	Bump checksum error counter before reporting to ZED	2022-12-02 17:42:22 -08:00
vdev_rebuild.c	Increase default zfs_rebuild_vdev_limit to 64MB	2023-01-27 10:02:24 -08:00
vdev_removal.c	Bypass metaslab throttle for removal allocations	2022-12-09 10:48:33 -08:00
vdev_root.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
vdev_trim.c	Propagate extent_bytes change to autotrim thread	2022-10-28 10:16:31 -07:00
vdev.c	Configure zed's diagnosis engine with vdev properties	2023-01-23 13:14:25 -08:00
zap_leaf.c	Optimize microzaps	2022-10-20 11:57:15 -07:00
zap_micro.c	Add tunable to allow changing micro ZAP's max size	2023-01-10 13:41:54 -08:00
zap.c	Cleanup: Use NULL when doing NULL pointer comparisons	2023-01-12 16:00:37 -08:00
zcp_get.c	Cleanup: Address Clang's static analyzer's unused code complaints	2022-10-14 13:37:54 -07:00
zcp_global.c	OpenZFS 8600 - ZFS channel programs - snapshot	2018-02-08 15:29:24 -08:00
zcp_iter.c	module/*.ko: prune .data, global .rodata	2022-01-14 15:37:55 -08:00
zcp_set.c	Support setting user properties in a channel program	2020-02-14 13:41:42 -08:00
zcp_synctask.c	Add zfs.sync.snapshot_rename	2022-09-02 13:31:19 -07:00
zcp.c	ztest: update ztest_dmu_snapshot_create_destroy()	2023-01-10 13:27:48 -08:00
zfeature.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
zfs_byteswap.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
zfs_chksum.c	Add generic implementation handling and SHA2 impl	2023-03-02 13:52:21 -08:00
zfs_fm.c	Configure zed's diagnosis engine with vdev properties	2023-01-23 13:14:25 -08:00
zfs_fuid.c	Cleanup: Remove unneeded semicolons	2023-01-12 16:00:30 -08:00
zfs_impl.c	Add generic implementation handling and SHA2 impl	2023-03-02 13:52:21 -08:00
zfs_ioctl.c	Remove bad kmem_free() oversight from previous zfsdev_state_list patch	2023-03-01 13:20:53 -08:00
zfs_log.c	zfs_rename: support RENAME_* flags	2022-10-28 09:49:20 -07:00
zfs_onexit.c	zfs_onexit_add_cb: make action_handle point to a uintptr_t	2022-11-03 09:52:12 -07:00
zfs_quota.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
zfs_ratelimit.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
zfs_replay.c	Suppress Clang static analyzer complaint in zfs_replay_create()	2023-02-14 11:05:41 -08:00
zfs_rlock.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
zfs_sa.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
zfs_vnops.c	Linux: use filemap_range_has_page()	2023-02-14 11:04:34 -08:00
zil.c	Do not hold spa_config in ZIL while blocked on IO	2023-03-07 16:12:28 -08:00
zio_checksum.c	Add generic implementation handling and SHA2 impl	2023-03-02 13:52:21 -08:00
zio_compress.c	Skip memory allocation when compressing holes	2023-02-27 14:41:02 -08:00
zio_inject.c	Cleanup: Switch to strlcpy from strncpy	2022-09-27 16:35:29 -07:00
zio.c	Skip memory allocation when compressing holes	2023-02-27 14:41:02 -08:00
zle.c	Replace dead opensolaris.org license link	2022-07-11 14:16:13 -07:00
zrlock.c	Micro-optimize zrl_remove()	2022-11-29 09:26:03 -08:00
zthr.c	Switch from _Noreturn to __attribute__((noreturn))	2022-03-23 08:51:00 -07:00
zvol.c	Cleanup of dead code suggested by Clang Static Analyzer (#14380)	2023-01-17 09:57:12 -08:00