c/mirror_zfs
1
0
Fork 0
mirror of https://git.proxmox.com/git/mirror_zfs.git synced 2024-11-17 10:01:01 +03:00
Code Issues Packages Projects Releases Wiki Activity
b0cbc1aa9a
mirror_zfs/module/zfs
History
Alexander Motin b0cbc1aa9a
Use big transactions for small recordsize writes. 
When ZFS appends files in chunks bigger than recordsize, it borrows
buffer from ARC and fills it before opening transaction.  This
supposed to help in case of page faults to not hold transaction open
indefinitely.  The problem appears when recordsize is set lower than
default 128KB. Since each block is committed in separate transaction,
per-transaction overhead becomes significant, and what is even worse,
active use of of per-dataset and per-pool locks to protect space use
accounting for each transaction badly hurts the code SMP scalability.
The same transaction size limitation applies in case of file rewrite,
but without even excuse of buffer borrowing.

To address the issue, disable the borrowing mechanism if recordsize
is smaller than default and the write request is 4x bigger than it.
In such case writes up to 32MB are executed in single transaction,
that dramatically reduces overhead and lock contention.  Since the
borrowing mechanism is not used for file rewrites, and it was never
used by zvols, which seem to work fine, I don't think this change
should create significant problems, partially because in addition to
the borrowing mechanism there are also used pre-faults.

My tests with 4/8 threads writing several files same time on datasets
with 32KB recordsize in 1MB requests show reduction of CPU usage by
the user threads by 25-35%.  I would measure it in GB/s, but at that
block size we are now limited by the lock contention of single write
issue taskqueue, which is a separate problem we are going to work on.

Reviewed-by: Brian Atkinson <batkinson@lanl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by:	Alexander Motin <mav@FreeBSD.org>
Sponsored by:	iXsystems, Inc.
Closes #14964
2023-06-27 17:00:30 -07:00
..
abd.c Fix two abd_gang_add_gang() issues. 2023-05-05 09:17:55 -07:00
aggsum.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
arc.c Remove ARC/ZIO physdone callbacks. 2023-06-15 10:49:03 -07:00
blake3_zfs.c blake3: fix up bogus checksums in face of cpu migration 2023-05-01 17:21:27 -07:00
blkptr.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
bplist.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
bpobj.c Suppress Clang Static Analyzer warning in bpobj_enqueue() 2023-03-08 13:51:55 -08:00
bptree.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
bqueue.c Batch enqueue/dequeue for bqueue 2023-01-10 13:39:22 -08:00
brt.c Replace dead opensolaris.org license links 2023-03-14 14:44:01 -07:00
btree.c btree: Implement faster binary search algorithm 2023-05-26 10:03:12 -07:00
dataset_kstats.c zil: Add some more statistics. 2023-05-25 13:51:53 -07:00
dbuf_stats.c Revert "Reduce dbuf_find() lock contention" 2022-09-22 12:59:41 -07:00
dbuf.c Remove ARC/ZIO physdone callbacks. 2023-06-15 10:49:03 -07:00
ddt_zap.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
ddt.c Implementation of block cloning for ZFS 2023-03-10 11:59:53 -08:00
dmu_diff.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dmu_object.c zfs redact fails when dnodesize=auto 2023-02-16 09:23:39 -08:00
dmu_objset.c Remove ARC/ZIO physdone callbacks. 2023-06-15 10:49:03 -07:00
dmu_recv.c Finally drop long disabled vdev cache. 2023-06-09 12:40:55 -07:00
dmu_redact.c Cleanup: Remove constant comparisons reported by CodeQL 2023-03-08 13:51:46 -08:00
dmu_send.c zdb: add -B option to generate backup stream 2023-06-05 11:54:42 -07:00
dmu_traverse.c Fix prefetching of indirect blocks while destroying 2023-03-24 10:20:07 -07:00
dmu_tx.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
dmu_zfetch.c Introduce zfs_refcount_(add|remove)_few(). 2023-06-05 11:51:44 -07:00
dmu.c Remove ARC/ZIO physdone callbacks. 2023-06-15 10:49:03 -07:00
dnode_sync.c free_blocks(): Fix reports from 2016 PVS Studio FreeBSD report 2023-01-23 13:12:37 -08:00
dnode.c Revert "ZFS_IOC_COUNT_FILLED does unnecessary txg_wait_synced()" 2023-04-18 08:41:52 -07:00
dsl_bookmark.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
dsl_crypt.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
dsl_dataset.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
dsl_deadlist.c Add missing increment to dsl_deadlist_move_bpobj() 2023-03-06 15:28:26 -08:00
dsl_deleg.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dsl_destroy.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
dsl_dir.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
dsl_pool.c Move dmu_buf_rele() after dsl_dataset_sync_done() 2023-02-23 18:14:52 -07:00
dsl_prop.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
dsl_scan.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
dsl_synctask.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dsl_userhold.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
edonr_zfs.c Remove unused Edon-R variants 2023-03-14 15:59:58 -07:00
fm.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
gzip.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
hkdf.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
lz4_zfs.c Updated the lz4 decompressor 2022-01-07 10:36:49 -08:00
lz4.c lz4: Cherrypick fix for CVE-2021-3520 2022-01-12 16:14:36 -08:00
lzjb.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
metaslab.c Introduce zfs_refcount_(add|remove)_few(). 2023-06-05 11:51:44 -07:00
mmp.c Allow MMP to bypass waiting for other threads 2023-04-19 13:22:59 -07:00
multilist.c Cleanup: Specify unsignedness on things that should not be signed 2022-09-27 16:42:41 -07:00
objlist.c Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
pathname.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
range_tree.c btree: Implement faster binary search algorithm 2023-05-26 10:03:12 -07:00
refcount.c Switch refcount tracking from lists to AVL-trees. 2023-06-14 08:02:27 -07:00
rrwlock.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
sa.c Cleanup: Remove constant comparisons reported by CodeQL 2023-03-08 13:51:46 -08:00
sha2_zfs.c Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
skein_zfs.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
spa_checkpoint.c Cleanup: 64-bit kernel module parameters should use fixed width types 2022-10-13 10:03:29 -07:00
spa_config.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
spa_errlog.c Teach zpool scrub to scrub only blocks in error log 2023-05-18 11:59:42 -07:00
spa_history.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
spa_log_spacemap.c Address warnings about possible division by zero from clangsa 2022-11-03 09:58:14 -07:00
spa_misc.c Another set of vdev queue optimizations. 2023-06-27 09:09:48 -07:00
spa_stats.c Cleanup: Specify unsignedness on things that should not be signed 2022-09-27 16:42:41 -07:00
spa.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
space_map.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
space_reftree.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
THIRDPARTYLICENSE.cityhash OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
THIRDPARTYLICENSE.cityhash.descrip OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
txg.c Another set of vdev queue optimizations. 2023-06-27 09:09:48 -07:00
uberblock.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
unique.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_draid_rand.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_draid.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
vdev_indirect_births.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
vdev_indirect_mapping.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
vdev_indirect.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
vdev_initialize.c Add the ability to uninitialize 2023-05-18 10:02:20 -07:00
vdev_label.c Store the L2ARC device ashift in the vdev label 2023-06-14 08:01:17 -07:00
vdev_mirror.c Improve too large physical ashift handling 2022-09-08 10:30:53 -07:00
vdev_missing.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_queue.c Another set of vdev queue optimizations. 2023-06-27 09:09:48 -07:00
vdev_raidz_math_aarch64_neon_common.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_aarch64_neon.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_aarch64_neonx2.c Fix Clang 15 compilation errors 2022-11-30 13:46:26 -08:00
vdev_raidz_math_avx2.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_avx512bw.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_avx512f.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_impl.h Cleanup Raid-Z Typo fixes 2022-09-06 09:43:21 -07:00
vdev_raidz_math_powerpc_altivec_common.h Linux ppc64le ieee128 compat: Do not redefine __asm on external headers 2023-01-13 10:58:58 -08:00
vdev_raidz_math_powerpc_altivec.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_scalar.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_sse2.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_ssse3.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math.c Miscellaneous FreBSD compilation bugfixes 2023-04-06 10:35:02 -07:00
vdev_raidz.c Bump checksum error counter before reporting to ZED 2022-12-02 17:42:22 -08:00
vdev_rebuild.c Increase default zfs_rebuild_vdev_limit to 64MB 2023-01-27 10:02:24 -08:00
vdev_removal.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
vdev_root.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_trim.c Fix short-lived txg caused by autotrim 2023-03-28 08:43:41 -07:00
vdev.c Another set of vdev queue optimizations. 2023-06-27 09:09:48 -07:00
zap_leaf.c Cleanup: Remove constant comparisons reported by CodeQL 2023-03-08 13:51:46 -08:00
zap_micro.c btree: Implement faster binary search algorithm 2023-05-26 10:03:12 -07:00
zap.c Cleanup: Use NULL when doing NULL pointer comparisons 2023-01-12 16:00:37 -08:00
zcp_get.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
zcp_global.c OpenZFS 8600 - ZFS channel programs - snapshot 2018-02-08 15:29:24 -08:00
zcp_iter.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
zcp_set.c Support setting user properties in a channel program 2020-02-14 13:41:42 -08:00
zcp_synctask.c Add zfs.sync.snapshot_rename 2022-09-02 13:31:19 -07:00
zcp.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
zfeature.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_byteswap.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_chksum.c Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
zfs_fm.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
zfs_fuid.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
zfs_impl.c Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
zfs_ioctl.c Teach zpool scrub to scrub only blocks in error log 2023-05-18 11:59:42 -07:00
zfs_log.c Fix VERIFY(!zil_replaying(zilog, tx)) panic 2023-04-17 16:42:09 -07:00
zfs_onexit.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
zfs_quota.c Implementation of block cloning for ZFS 2023-03-10 11:59:53 -08:00
zfs_ratelimit.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_replay.c Linux 6.3 compat: idmapped mount API changes 2023-04-10 14:15:36 -07:00
zfs_rlock.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_sa.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_vnops.c Use big transactions for small recordsize writes. 2023-06-27 17:00:30 -07:00
zil.c Fix memory leak in zil_parse(). 2023-06-17 19:51:37 -07:00
zio_checksum.c Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
zio_compress.c Skip memory allocation when compressing holes 2023-02-27 14:41:02 -08:00
zio_inject.c Cleanup: Switch to strlcpy from strncpy 2022-09-27 16:35:29 -07:00
zio.c Remove ARC/ZIO physdone callbacks. 2023-06-15 10:49:03 -07:00
zle.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zrlock.c Micro-optimize zrl_remove() 2022-11-29 09:26:03 -08:00
zthr.c Switch from _Noreturn to __attribute__((noreturn)) 2022-03-23 08:51:00 -07:00
zvol.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00