Allan Jude b6916f995e ARC: parallel eviction ... On systems with enormous amounts of memory, the single arc_evict thread can become a bottleneck if reads and writes are stuck behind it, waiting for old data to be evicted before new data can take its place. This commit adds support for evicting from multiple ARC lists in parallel, by farming the evict work out to some number of threads and then accumulating their results. A new tuneable, zfs_arc_evict_threads, sets the number of threads. By default, it will scale based on the number of CPUs. Sponsored-by: Expensify, Inc. Sponsored-by: Klara, Inc. Reviewed-by: Alexander Motin <mav@FreeBSD.org> Reviewed-by: Youzhong Yang <youzhong@gmail.com> Signed-off-by: Allan Jude <allan@klarasystems.com> Signed-off-by: Mateusz Piotrowski <mateusz.piotrowski@klarasystems.com> Signed-off-by: Alexander Stetsenko <alex.stetsenko@klarasystems.com> Signed-off-by: Rob Norris <rob.norris@klarasystems.com> Co-authored-by: Rob Norris <rob.norris@klarasystems.com> Co-authored-by: Mateusz Piotrowski <mateusz.piotrowski@klarasystems.com> Co-authored-by: Alexander Stetsenko <alex.stetsenko@klarasystems.com> Closes #16486		2025-05-14 10:38:32 -04:00
..
abd.c	Export correct symbols for Lustre Direct I/O	2025-04-24 13:55:21 -04:00
aggsum.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
arc.c	ARC: parallel eviction	2025-05-14 10:38:32 -04:00
blake3_zfs.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
blkptr.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
bplist.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
bpobj.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
bptree.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
bqueue.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
brt.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
btree.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dataset_kstats.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dbuf_stats.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dbuf.c	ARC: Notify dbuf cache about target size reduction	2025-05-14 10:34:14 -04:00
ddt_log.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
ddt_stats.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
ddt_zap.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
ddt.c	Handle interaction between gang blocks, copies, and FDT.	2025-04-21 11:26:30 -04:00
dmu_diff.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dmu_direct.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
dmu_object.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dmu_objset.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
dmu_recv.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
dmu_redact.c	dmu_tx: rename dmu_tx_assign() flags from TXG_* to DMU_TX_* (#17143)	2025-03-18 16:04:22 -07:00
dmu_send.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dmu_traverse.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
dmu_tx.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
dmu_zfetch.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
dmu.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
dnode_sync.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
dnode.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
dsl_bookmark.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dsl_crypt.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dsl_dataset.c	cred: properly pass and test creds on other threads (#17273)	2025-04-29 16:27:48 -07:00
dsl_deadlist.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dsl_deleg.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dsl_destroy.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dsl_dir.c	cred: properly pass and test creds on other threads (#17273)	2025-04-29 16:27:48 -07:00
dsl_pool.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dsl_prop.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
dsl_scan.c	Fix race between resilver wait and offline/detach	2025-05-02 15:19:24 -07:00
dsl_synctask.c	txg: generalise txg_wait_synced_sig() to txg_wait_synced_flags() (#17284)	2025-05-02 15:29:50 -07:00
dsl_userhold.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
edonr_zfs.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
fm.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
gzip.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
hkdf.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
lz4_zfs.c	SPDX: license tags: BSD-2-Clause	2025-03-13 17:56:46 -07:00
lz4.c	SPDX: license tags: BSD-2-Clause	2025-03-13 17:56:46 -07:00
lzjb.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
metaslab.c	metaslab_alloc: make hint BP and DVA const (#17324)	2025-05-12 10:52:46 -07:00
mmp.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
multilist.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
objlist.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
pathname.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
range_tree.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
refcount.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
rrwlock.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
sa.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
sha2_zfs.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
skein_zfs.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
spa_checkpoint.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
spa_config.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
spa_errlog.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
spa_history.c	dmu_tx: rename dmu_tx_assign() flags from TXG_* to DMU_TX_* (#17143)	2025-03-18 16:04:22 -07:00
spa_log_spacemap.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
spa_misc.c	Unified allocation throttling (#17020)	2025-03-24 09:25:01 -07:00
spa_stats.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
spa.c	spa: clear checkpoint information during retry	2025-05-11 12:49:38 -04:00
space_map.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
space_reftree.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
THIRDPARTYLICENSE.cityhash	OpenZFS 8484 - Implement aggregate sum and use for arc counters	2018-06-06 09:35:59 -07:00
THIRDPARTYLICENSE.cityhash.descrip	OpenZFS 8484 - Implement aggregate sum and use for arc counters	2018-06-06 09:35:59 -07:00
txg.c	txg: generalise txg_wait_synced_sig() to txg_wait_synced_flags() (#17284)	2025-05-02 15:29:50 -07:00
uberblock.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
unique.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_draid_rand.c	SPDX: license tags: LicenseRef-OpenZFS-ThirdParty-PublicDomain	2025-03-13 17:57:31 -07:00
vdev_draid.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
vdev_file.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
vdev_indirect_births.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_indirect_mapping.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_indirect.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
vdev_initialize.c	dmu_tx: rename dmu_tx_assign() flags from TXG_* to DMU_TX_* (#17143)	2025-03-18 16:04:22 -07:00
vdev_label.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_mirror.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
vdev_missing.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
vdev_queue.c	Unified allocation throttling (#17020)	2025-03-24 09:25:01 -07:00
vdev_raidz_math_aarch64_neon_common.h	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_aarch64_neon.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_aarch64_neonx2.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_avx2.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_avx512bw.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_avx512f.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_impl.h	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_powerpc_altivec_common.h	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_powerpc_altivec.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_scalar.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_sse2.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math_ssse3.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz_math.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
vdev_raidz.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
vdev_rebuild.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
vdev_removal.c	Fix lock reversal on device removal cancel	2025-04-01 09:31:24 -04:00
vdev_root.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
vdev_trim.c	dmu_tx: rename dmu_tx_assign() flags from TXG_* to DMU_TX_* (#17143)	2025-03-18 16:04:22 -07:00
vdev.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
zap_leaf.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zap_micro.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zap.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zcp_get.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zcp_global.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zcp_iter.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zcp_set.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zcp_synctask.c	cred: properly pass and test creds on other threads (#17273)	2025-04-29 16:27:48 -07:00
zcp.c	txg: generalise txg_wait_synced_sig() to txg_wait_synced_flags() (#17284)	2025-05-02 15:29:50 -07:00
zfeature.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zfs_byteswap.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zfs_chksum.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zfs_debug_common.c	nvlist: Add nvlist_snprintf() and zfs_dbgmsg_nvlist()	2025-04-18 09:22:16 -04:00
zfs_fm.c	zed: Ensure spare activation after kernel-initiated device removal	2025-03-28 15:48:38 -04:00
zfs_fuid.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zfs_impl.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zfs_ioctl.c	Show default quotas in zfs userspace tools	2025-04-03 10:36:45 -07:00
zfs_log.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
zfs_onexit.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zfs_quota.c	Show default quotas in zfs userspace tools	2025-04-03 10:36:45 -07:00
zfs_ratelimit.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zfs_replay.c	dmu_tx: rename dmu_tx_assign() flags from TXG_* to DMU_TX_* (#17143)	2025-03-18 16:04:22 -07:00
zfs_rlock.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zfs_sa.c	dmu_tx: rename dmu_tx_assign() flags from TXG_* to DMU_TX_* (#17143)	2025-03-18 16:04:22 -07:00
zfs_vnops.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00
zfs_znode.c	Add default user/group/project quota properties	2025-04-03 10:35:22 -07:00
zil.c	dmu_tx: rename dmu_tx_assign() flags from TXG_* to DMU_TX_* (#17143)	2025-03-18 16:04:22 -07:00
zio_checksum.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zio_compress.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zio_inject.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zio.c	Implement allocation size ranges and use for gang leaves (#17111)	2025-05-02 15:32:18 -07:00
zle.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zrlock.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zthr.c	SPDX: license tags: CDDL-1.0	2025-03-13 17:56:27 -07:00
zvol.c	Wire O_DIRECT also to Uncached I/O (#17218)	2025-05-13 14:26:55 -07:00