c/mirror_zfs
1
0
Fork 0
mirror of https://git.proxmox.com/git/mirror_zfs.git synced 2025-02-22 15:04:22 +03:00
Code Issues Packages Projects Releases Wiki Activity
478c5b4f3f
mirror_zfs/module/zfs
History
Rob Norris 12a031a3f5 spa: make read/write queues configurable 
We are finding that as customers get larger and faster machines
(hundreds of cores, large NVMe-backed pools) they keep hitting
relatively low performance ceilings. Our profiling work almost always
finds that they're running into bottlenecks on the SPA IO taskqs.
Unfortunately there's often little we can advise at that point, because
there's very few ways to change behaviour without patching.

This commit adds two load-time parameters `zio_taskq_read` and
`zio_taskq_write` that can configure the READ and WRITE IO taskqs
directly.

This achieves two goals: it gives operators (and those that support
them) a way to tune things without requiring a custom build of OpenZFS,
which is often not possible, and it lets us easily try different config
variations in a variety of environments to inform the development of
better defaults for these kind of systems.

Because tuning the IO taskqs really requires a fairly deep understanding
of how IO in ZFS works, and generally isn't needed without a pretty
serious workload and an ability to identify bottlenecks, only minimal
documentation is provided. Its expected that anyone using this is going
to have the source code there as well.

Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
2023-12-22 13:24:58 -08:00
..
abd.c Fix two abd_gang_add_gang() issues. 2023-06-02 11:17:11 -07:00
aggsum.c More aggsum optimizations 2021-06-09 13:05:34 -07:00
arc.c Fix the L2ARC write size calculating logic (2) 2023-06-26 13:59:36 -07:00
blkptr.c Add zstd support to zfs 2020-08-20 10:30:06 -07:00
bplist.c Fast Clone Deletion 2019-07-26 10:54:14 -07:00
bpobj.c Prefetch on deadlists merge 2023-03-02 14:37:07 -08:00
bptree.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
bqueue.c zfs recv hangs if max recordsize is less than received recordsize 2022-09-19 09:39:07 -07:00
btree.c Optimize microzaps 2023-03-02 14:37:07 -08:00
dataset_kstats.c Introduce write-mostly sums 2021-06-09 13:05:34 -07:00
dbuf_stats.c Revert "Reduce dbuf_find() lock contention" 2022-09-21 13:15:51 -07:00
dbuf.c Optimize the is_l2cacheable functions 2023-03-07 09:07:58 -08:00
ddt_zap.c Refactor dnode dirty context from dbuf_dirty 2020-02-26 16:09:17 -08:00
ddt.c Tinker with slop space accounting with dedup 2021-09-14 12:38:05 -07:00
dmu_diff.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
dmu_object.c Introduce CPU_SEQID_UNSTABLE 2020-11-02 11:51:12 -08:00
dmu_objset.c Optimize the is_l2cacheable functions 2023-03-07 09:07:58 -08:00
dmu_recv.c Fix raw receive with different indirect block size. 2023-07-20 08:59:14 -07:00
dmu_redact.c Fix incorrect size given to bqueue_enqueue() call in dmu_redact.c 2022-09-21 13:15:51 -07:00
dmu_send.c Fix NULL pointer dereference when doing concurrent 'send' operations 2023-05-31 17:02:38 -07:00
dmu_traverse.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
dmu_tx.c Add dmu_tx_hold_append() interface 2023-05-11 09:08:08 -07:00
dmu_zfetch.c System-wide speculative prefetch limit. 2023-03-02 14:37:07 -08:00
dmu.c Additional limits on hole reporting 2023-03-29 10:40:49 -07:00
dnode_sync.c Report dnodes with faulty bonuslen 2022-02-16 17:58:55 -08:00
dnode.c dnode_is_dirty: check dnode and its data for dirtiness 2023-11-28 09:16:49 -08:00
dsl_bookmark.c Fix -Wattribute-warning in dsl layer 2022-07-27 13:38:56 -07:00
dsl_crypt.c Move dmu_buf_rele() after dsl_dataset_sync_done() 2023-03-01 09:36:19 -08:00
dsl_dataset.c Revert zfeature_active() to static 2023-03-01 09:36:19 -08:00
dsl_deadlist.c Move zap_attribute_t to the heap in dsl_deadlist_merge 2023-06-01 08:34:52 -07:00
dsl_deleg.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
dsl_destroy.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
dsl_dir.c Avoid null pointer dereference in dsl_fs_ss_limit_check() 2022-12-01 12:39:43 -08:00
dsl_pool.c Move dmu_buf_rele() after dsl_dataset_sync_done() 2023-03-01 09:36:19 -08:00
dsl_prop.c dsl_prop_known_index(): check for invalid prop 2022-11-08 10:16:21 -08:00
dsl_scan.c Increase default zfs_scan_vdev_limit to 16MB 2023-04-24 12:55:07 -07:00
dsl_synctask.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
dsl_userhold.c Replace sprintf()->snprintf() and strcpy()->strlcpy() 2020-06-07 11:42:12 -07:00
edonr_zfs.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
fm.c fm_fmri_hc_create() must call va_end() before returning 2022-12-01 12:39:42 -08:00
gzip.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
hkdf.c
lz4.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
lzjb.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
Makefile.in Fix Clang 15 compilation errors 2023-06-05 18:25:57 -07:00
metaslab.c Stop ganging due to past vdev write errors 2022-11-01 12:36:25 -07:00
mmp.c Allow MMP to bypass waiting for other threads 2023-04-24 12:55:07 -07:00
multilist.c Optimize small random numbers generation 2021-09-14 12:10:17 -07:00
objlist.c Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
pathname.c Replace ZFS on Linux references with OpenZFS 2020-10-08 20:10:13 -07:00
range_tree.c Several sorted scrub optimizations 2022-07-26 10:10:37 -07:00
refcount.c Export minimal zfs_refcount interfaces 2022-04-06 10:29:00 -07:00
rrwlock.c Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
sa.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
sha256.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
skein_zfs.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
spa_boot.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
spa_checkpoint.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
spa_config.c zed: add hotplug support for spare vdevs 2023-03-27 11:32:09 -07:00
spa_errlog.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
spa_history.c Annotated dprintf as printf-like 2021-06-24 13:12:36 -07:00
spa_log_spacemap.c Address warnings about possible division by zero from clangsa 2022-12-01 12:39:43 -08:00
spa_misc.c Allow MMP to bypass waiting for other threads 2023-04-24 12:55:07 -07:00
spa_stats.c Remove pool io kstats 2021-06-10 10:50:16 -07:00
spa.c spa: make read/write queues configurable 2023-12-22 13:24:58 -08:00
space_map.c Optimize small random numbers generation 2021-09-14 12:10:17 -07:00
space_reftree.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
THIRDPARTYLICENSE.cityhash OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
THIRDPARTYLICENSE.cityhash.descrip OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
txg.c Optimize txg_kick() process (#12274) 2022-09-21 16:12:14 -07:00
uberblock.c MMP interval and fail_intervals in uberblock 2019-03-21 12:47:57 -07:00
unique.c Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
vdev_cache.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
vdev_draid_rand.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_draid.c Improve too large physical ashift handling 2022-09-21 13:15:15 -07:00
vdev_indirect_births.c Fixes: #8934 Large kmem_alloc 2019-07-10 15:54:49 -07:00
vdev_indirect_mapping.c Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
vdev_indirect.c Cleanup: Replace oldstyle struct hack with C99 flexible array members 2023-09-20 10:10:41 -07:00
vdev_initialize.c Add the ability to uninitialize 2023-05-26 10:09:04 -07:00
vdev_label.c Report ashift of L2ARC devices in zdb 2023-12-21 11:16:30 -08:00
vdev_mirror.c Improve too large physical ashift handling 2022-09-21 13:15:15 -07:00
vdev_missing.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
vdev_queue.c Avoid vq_lock drop in vdev_queue_aggregate() 2021-09-14 14:31:22 -07:00
vdev_raidz_math_aarch64_neon_common.h FreeBSD: fix the build with Clang 11 2020-08-17 15:40:17 -07:00
vdev_raidz_math_aarch64_neon.c Linux 5.0 compat: SIMD compatibility 2019-07-12 09:31:20 -07:00
vdev_raidz_math_aarch64_neonx2.c vdev_raidz_math_aarch64_neonx2.c: suppress diagnostic only for GCC 2022-12-09 12:07:38 -08:00
vdev_raidz_math_avx2.c FreeBSD: fix the build with Clang 11 2020-08-17 15:40:17 -07:00
vdev_raidz_math_avx512bw.c Refactor ccompile.h to not include system headers 2020-07-25 20:09:50 -07:00
vdev_raidz_math_avx512f.c FreeBSD: fix the build with Clang 11 2020-08-17 15:40:17 -07:00
vdev_raidz_math_impl.h Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_raidz_math_powerpc_altivec_common.h Linux ppc64le ieee128 compat: Do not redefine __asm on external headers 2023-01-19 12:50:42 -08:00
vdev_raidz_math_powerpc_altivec.c Prefix zfs internal endian checks with _ZFS 2020-07-28 13:02:49 -07:00
vdev_raidz_math_scalar.c Use fallthrough macro 2021-11-02 09:50:30 -07:00
vdev_raidz_math_sse2.c FreeBSD: fix the build with Clang 11 2020-08-17 15:40:17 -07:00
vdev_raidz_math_ssse3.c Refactor ccompile.h to not include system headers 2020-07-25 20:09:50 -07:00
vdev_raidz_math.c Initialize parity blocks before RAID-Z reconstruction benchmarking 2021-09-14 14:32:16 -07:00
vdev_raidz.c Improve too large physical ashift handling 2022-09-21 13:15:15 -07:00
vdev_rebuild.c Increase default zfs_rebuild_vdev_limit to 64MB 2023-04-24 12:55:07 -07:00
vdev_removal.c module: zfs: vdev_removal: remove unused num_indirect 2022-12-01 12:39:44 -08:00
vdev_root.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_trim.c Zpool can start allocating from metaslab before TRIMs have completed 2023-11-29 12:56:23 -08:00
vdev.c Probe vdevs before marking removed 2023-05-26 10:08:04 -07:00
zap_leaf.c Optimize microzaps 2023-03-02 14:37:07 -08:00
zap_micro.c Optimize microzaps 2023-03-02 14:37:07 -08:00
zap.c Remove unneeded "extern inline" function declarations 2022-02-16 17:58:56 -08:00
zcp_get.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
zcp_global.c
zcp_iter.c Fix typos in module/zfs/ 2019-09-02 17:56:41 -07:00
zcp_set.c Support setting user properties in a channel program 2020-02-14 13:41:42 -08:00
zcp_synctask.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
zcp.c Fix too few arguments to formatting function 2022-12-01 12:39:43 -08:00
zfeature.c Throw const on some strings 2020-10-02 17:44:10 -07:00
zfs_byteswap.c Mark functions as static 2020-06-18 12:20:38 -07:00
zfs_fm.c Remove an unused variable 2022-12-01 12:39:43 -08:00
zfs_fuid.c Fix regression in POSIX mode behavior 2021-03-19 22:50:46 -07:00
zfs_ioctl.c Add the ability to uninitialize 2023-05-26 10:09:04 -07:00
zfs_log.c Add Module Parameter Regarding Log Size Limit 2022-09-21 16:12:14 -07:00
zfs_onexit.c file reference counts can get corrupted 2021-09-14 12:37:38 -07:00
zfs_quota.c File incorrectly zeroed when receiving incremental stream that toggles -L 2020-06-09 10:41:01 -07:00
zfs_ratelimit.c
zfs_replay.c Use fallthrough macro 2021-11-02 09:50:30 -07:00
zfs_rlock.c Add a "try" operation for range locks 2020-07-06 11:53:31 -07:00
zfs_sa.c Extending FreeBSD UIO Struct 2021-01-20 21:27:30 -08:00
zfs_vnops.c Linux: use filemap_range_has_page() 2023-06-05 10:59:02 -07:00
zil.c Fix memory leak in zil_parse(). 2023-06-26 13:58:46 -07:00
zio_checksum.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
zio_compress.c module: zfs: fix unused, remove argsused 2022-02-16 17:58:56 -08:00
zio_inject.c Optimize small random numbers generation 2021-09-14 12:10:17 -07:00
zio.c Remove single parent assertion from zio_nowait(). 2023-06-02 11:17:11 -07:00
zle.c Add include files for prototypes 2020-06-18 12:21:25 -07:00
zrlock.c Micro-optimize zrl_remove() 2023-03-02 14:37:07 -08:00
zthr.c Avoid memory allocations in the ARC eviction thread 2022-02-03 15:30:52 -08:00
zvol.c Add Module Parameter Regarding Log Size Limit 2022-09-21 16:12:14 -07:00