mirror_zfs/include/sys
Alexander Motin 46d6f1fe56
DDT: Move logs searches out of the lock
Postponing entry removal from the DDT log in case of hit till later
single-threaded sync stage allows to make ddl_tree stable during
multi-threaded ZIO processing stage.  It allows to drop the DDT lock
before the search instead of after, reducing the contention a lot.

Actually ddt_log_update_entry() was already handling the case of
entry present in the active log, so we only need to remove it from
flushing log, if the entry happen to be there.

My tests with parallel 4KB block writes show throughput increase
from 480MB/s (122K blocks/s) to 827MB/s (212K blocks/s), even
though still limited by the global DDT lock contention.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Alexander Motin <alexander.motin@TrueNAS.com>
Closes #18044
2025-12-15 09:17:04 -08:00
..
crypto icp: remove global icp includes 2025-11-12 10:03:51 -08:00
fm Detect a slow raidz child during reads 2025-09-10 15:25:03 -07:00
fs Add knob to disable slow io notifications 2025-11-11 10:42:17 -08:00
lua SPDX: license tags: MIT 2025-03-13 17:56:54 -07:00
sysevent SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zstd SPDX: license tags: BSD-3-Clause 2025-03-13 17:56:50 -07:00
abd_impl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
abd.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
aggsum.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
arc_impl.h enforce arc_dnode_limit 2025-07-21 10:32:01 -07:00
arc.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
asm_linkage.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
avl_impl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
avl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
bitmap.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
bitops.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
blake3.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
blkptr.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
bplist.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
bpobj.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
bptree.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
bqueue.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
brt_impl.h BRT: Fix ranges to blocks conversion math 2025-11-10 13:58:39 -08:00
brt.h Add BRT support to zpool prefetch command 2025-11-10 16:16:22 -08:00
btree.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
dataset_kstats.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
dbuf.h Implement physical rewrites 2025-08-06 10:36:56 -07:00
ddt_impl.h DDT: Move logs searches out of the lock 2025-12-15 09:17:04 -08:00
ddt.h DDT: Move logs searches out of the lock 2025-12-15 09:17:04 -08:00
dmu_impl.h dnode: remove dn_dirtyctx and dnode_dirtycontext 2025-08-21 06:05:38 -07:00
dmu_objset.h Relax special_small_blocks restrictions 2025-07-02 11:11:37 -07:00
dmu_recv.h cred: properly pass and test creds on other threads (#17273) 2025-04-29 16:27:48 -07:00
dmu_redact.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
dmu_send.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
dmu_traverse.h Allow physical rewrite without logical 2025-08-06 10:36:07 -07:00
dmu_tx.h dmu_tx: break tx assign/wait when pool suspends 2025-05-28 10:28:51 -07:00
dmu_zfetch.h Wire O_DIRECT also to Uncached I/O (#17218) 2025-05-13 14:26:55 -07:00
dmu.h Fix caching of DDT log and BRT 2025-10-30 16:28:28 -07:00
dnode.h dnode: remove dn_dirtyctx and dnode_dirtycontext 2025-08-21 06:05:38 -07:00
dsl_bookmark.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
dsl_crypt.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
dsl_dataset.h dsl_dataset: rename dmu_objset_clone* to dsl_dataset_clone* 2025-06-10 14:52:43 -07:00
dsl_deadlist.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
dsl_deleg.h ZFS allow send:encrypted 2025-09-12 09:53:31 -07:00
dsl_destroy.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
dsl_dir.h cred: properly pass and test creds on other threads (#17273) 2025-04-29 16:27:48 -07:00
dsl_pool.h Unified allocation throttling (#17020) 2025-03-24 09:25:01 -07:00
dsl_prop.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
dsl_scan.h Improve async destroy processing timing 2025-12-11 18:46:08 -08:00
dsl_synctask.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
dsl_userhold.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
edonr.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
efi_partition.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
frame.h Linux build: silence objtool warnings 2025-06-04 17:40:09 -07:00
hkdf.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
metaslab_impl.h range_tree: Provide more debug details upon unexpected add/remove 2025-07-31 10:44:42 -04:00
metaslab.h metaslab: don't pass whole zio to throttle reserve APIs 2025-07-04 23:22:22 -04:00
mmp.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
mntent.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
multilist.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
nvpair_impl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
nvpair.h nvlist: Add nvlist_snprintf() and zfs_dbgmsg_nvlist() 2025-04-18 09:22:16 -04:00
objlist.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
pathname.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
qat.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
range_tree.h range_tree: drop duplicate zfs_ prefix from rs_set_fill_raw 2025-09-29 16:38:52 -07:00
rrwlock.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
sa_impl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
sa.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
sha2.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
skein.h SPDX: license tags: LicenseRef-OpenZFS-ThirdParty-PublicDomain 2025-03-13 17:57:31 -07:00
spa_checkpoint.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
spa_checksum.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
spa_impl.h Add interface to interface spa_get_worst_case_min_alloc() function 2025-09-25 09:35:35 -07:00
spa_log_spacemap.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
spa.h spa_misc: add an API for spa_namespace_lock 2025-11-10 14:23:39 -08:00
space_map.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
space_reftree.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
sysevent.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
txg_impl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
txg.h txg_wait_synced_flags: add TXG_WAIT_SUSPEND flag to not wait if pool suspended 2025-05-28 10:27:46 -07:00
u8_textprep_data.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
u8_textprep.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
uberblock_impl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
uberblock.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
uio_impl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
unique.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
uuid.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
vdev_disk.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
vdev_draid.h Implement allocation size ranges and use for gang leaves (#17111) 2025-05-02 15:32:18 -07:00
vdev_file.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
vdev_impl.h Add knob to disable slow io notifications 2025-11-11 10:42:17 -08:00
vdev_indirect_births.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
vdev_indirect_mapping.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
vdev_initialize.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
vdev_raidz_impl.h Detect a slow raidz child during reads 2025-09-10 15:25:03 -07:00
vdev_raidz.h Detect a slow raidz child during reads 2025-09-10 15:25:03 -07:00
vdev_rebuild.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
vdev_removal.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
vdev_trim.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
vdev.h Fix dynamic gang block headers on raidz and mirror devices 2025-08-06 09:50:58 -07:00
xvattr.h Add support for FreeBSD's Solaris style extended attribute interface 2025-07-30 09:49:43 -07:00
zap_impl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zap_leaf.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zap.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zcp_global.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zcp_iter.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zcp_prop.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zcp_set.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zcp.h cred: properly pass and test creds on other threads (#17273) 2025-04-29 16:27:48 -07:00
zfeature.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_acl.h libspl: move vattr and xvattr definitions from zfs_context.h 2025-11-12 10:00:24 -08:00
zfs_bootenv.h libzpool: add BE_POSIX_VENDOR for userspace bootenv 2025-11-12 10:03:07 -08:00
zfs_chksum.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_context.h zfs_context: remove duplicate includes 2025-11-12 10:04:03 -08:00
zfs_debug.h debug: move all of the debug bits out of the spl 2025-11-12 10:02:51 -08:00
zfs_delay.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_file.h Ensure 64-bit off_t is used in user space instead of loff_t 2025-12-10 09:45:39 -08:00
zfs_fuid.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_impl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_ioctl_impl.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_ioctl.h zinject: Introduce ready delay fault injection 2025-10-01 12:17:13 -07:00
zfs_onexit.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_project.h Linux 6.17 compat: Fix broken projectquota on 6.17 2025-11-05 16:22:03 -08:00
zfs_quota.h Show default quotas in zfs userspace tools 2025-04-03 10:36:45 -07:00
zfs_racct.h Wire O_DIRECT also to Uncached I/O (#17218) 2025-05-13 14:26:55 -07:00
zfs_ratelimit.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_refcount.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_rlock.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_sa.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_stat.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_sysfs.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_vfsops.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zfs_vnops.h Introduce zfs rewrite subcommand (#17246) 2025-05-12 10:22:17 -07:00
zfs_znode.h Prefer VERIFY0(n) over VERIFY(n == 0) 2025-08-07 11:40:59 -07:00
zil_impl.h ZIL: Make allocations more flexible 2025-08-14 08:50:17 -07:00
zil.h ZIL: add zil_commit_flags() to make honouring failmode= optional 2025-08-08 16:43:33 -07:00
zio_checksum.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zio_compress.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zio_crypt.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zio_impl.h ZIO: ZIO_STAGE_DDT_WRITE is a blocking stage 2025-12-10 19:51:53 -05:00
zio.h zinject: Introduce ready delay fault injection 2025-10-01 12:17:13 -07:00
zrlock.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zthr.h SPDX: license tags: CDDL-1.0 2025-03-13 17:56:27 -07:00
zvol_impl.h zvol_remove_minors_impl: remove all async fallbacks 2025-08-19 10:06:47 -07:00
zvol.h zvol: reject suspend attempts when zvol is shutting down 2025-09-03 11:13:09 -07:00