mirror_zfs/module/zfs
Alexander Motin 1a5414ba2f BRT: More optimizations after per-vdev splitting
- With both pending and current AVL-trees being per-vdev and having
effectively identical comparison functions (pending tree compared
also birth time, but I don't believe it is possible for them to be
different for the same offset within one transaction group), it
makes no sense to move entries from one to another.  Instead inline
dramatically simplified brt_entry_addref() into brt_pending_apply().
It no longer requires bv_lock, since there is nothing concurrent
to it at the time.  And it does not need to search the tree for the
previous entries, since it is the same tree, we already have the
entry and we know it is unique.
 - Put brt_vdev_lookup() and brt_vdev_addref() into different tree
traversals to avoid false positives in the first due to the second
entcount modifications.  It saves dramatic amount of time when a
file cloned first time by not looking for non-existent ZAP entries.
 - Remove avl_is_empty(bv_tree) check from brt_maybe_exists().  I
don't think it is needed, since by the time all added entries are
already accounted in bv_entcount. The extra check must be producing
too many false positives for no reason.  Also we don't need bv_lock
there, since bv_entcount pointer must be table at this point, and
we don't care about false positive races here, while false negative
should be impossible, since all brt_vdev_addref() have already
completed by this point.  This dramatically reduces lock contention
on massive deletes of cloned blocks.  The only remaining one is
between multiple parallel free threads calling brt_entry_decref().
 - Do not update ZAP if net change for a block over the TXG was 0.
In combination with above it makes file move between datasets as
cheap operation as originally intended if it fits into one TXG.
 - Do not allocate vdevs on pool creation or import if it did not
have active block cloning. This allows to save a bit in few cases.
 - While here, add proper error handling in brt_load() on pool
import instead of assertions.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored by:	iXsystems, Inc.
Closes #16773
2024-11-21 08:24:37 -08:00
..
abd.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
aggsum.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
arc.c L2ARC: Stop rebuild before setting spa_final_txg 2024-11-21 08:24:37 -08:00
blake3_zfs.c blake3: fix up bogus checksums in face of cpu migration 2023-05-01 17:21:27 -07:00
blkptr.c compress: change zio_compress API to use ABDs 2024-08-22 16:22:24 -07:00
bplist.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
bpobj.c Provide macros for setting and getting blkptr birth times 2024-03-25 15:01:54 -07:00
bptree.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
bqueue.c Batch enqueue/dequeue for bqueue 2023-01-10 13:39:22 -08:00
brt.c BRT: More optimizations after per-vdev splitting 2024-11-21 08:24:37 -08:00
btree.c Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN. 2024-05-10 08:47:21 -07:00
dataset_kstats.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
dbuf_stats.c Consider dnode_t allocations in dbuf cache size accounting 2023-11-17 13:25:53 -08:00
dbuf.c BRT: Don't call brt_pending_remove() on holes/embedded 2024-11-21 08:24:37 -08:00
ddt_log.c Add DDT prune command 2024-09-04 14:17:02 -07:00
ddt_stats.c ddt: dedup log 2024-08-16 12:03:35 -07:00
ddt_zap.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
ddt.c Remove set but not used variable in ddt.c (#16522) 2024-09-10 12:46:50 -07:00
dmu_diff.c Simplify issig(). 2024-05-29 10:49:11 -07:00
dmu_direct.c Reduce dirty records memory usage 2024-11-04 16:46:39 -08:00
dmu_object.c Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN. 2024-05-10 08:47:21 -07:00
dmu_objset.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dmu_recv.c feature: large_microzap 2024-10-02 20:47:11 -07:00
dmu_redact.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dmu_send.c ZFS send should use spill block prefetched from send_reader_thread 2024-11-06 11:54:32 -08:00
dmu_traverse.c Provide macros for setting and getting blkptr birth times 2024-03-25 15:01:54 -07:00
dmu_tx.c Add warning for external consumers of dmu_tx_callback_register 2024-11-01 09:49:05 -07:00
dmu_zfetch.c Small fix to prefetch ranges aggregation 2024-04-09 16:14:04 -07:00
dmu.c Reduce dirty records memory usage 2024-11-04 16:46:39 -08:00
dnode_sync.c Reduce dirty records memory usage 2024-11-04 16:46:39 -08:00
dnode.c Linux: Report reclaimable memory to kernel as such (#16385) 2024-07-30 11:40:47 -07:00
dsl_bookmark.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_crypt.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_dataset.c dsl_dataset: put IO-inducing frees on the pool deadlist 2024-11-14 16:52:02 -08:00
dsl_deadlist.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_deleg.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_destroy.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_dir.c Support for longnames for files/directories (Linux part) 2024-10-01 13:40:27 -07:00
dsl_pool.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_prop.c snapdir: add 'disabled' value to make .zfs inaccessible 2024-10-02 09:12:02 -07:00
dsl_scan.c Defer resilver only when progress is above a threshold 2024-10-04 10:41:17 -07:00
dsl_synctask.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dsl_userhold.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
edonr_zfs.c Remove unused Edon-R variants 2023-03-14 15:59:58 -07:00
fm.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
gzip.c compress: change compression providers API to use ABDs 2024-08-22 16:22:24 -07:00
hkdf.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
lz4_zfs.c compress: change compression providers API to use ABDs 2024-08-22 16:22:24 -07:00
lz4.c lz4: Cherrypick fix for CVE-2021-3520 2022-01-12 16:14:36 -08:00
lzjb.c compress: change compression providers API to use ABDs 2024-08-22 16:22:24 -07:00
metaslab.c Remove extra newline from spa_set_allocator(). 2024-09-17 13:15:42 -07:00
mmp.c vdev probe to slow disk can stall mmp write checker 2024-04-29 14:35:53 -07:00
multilist.c L2ARC: Relax locking during write 2024-04-09 16:23:19 -07:00
objlist.c Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
pathname.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
range_tree.c btree: Implement faster binary search algorithm 2023-05-26 10:03:12 -07:00
refcount.c Switch refcount tracking from lists to AVL-trees. 2023-06-14 08:02:27 -07:00
rrwlock.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
sa.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
sha2_zfs.c Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
skein_zfs.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
spa_checkpoint.c RAID-Z expansion feature 2023-11-08 10:19:41 -08:00
spa_config.c Add mutex_enter_interruptible() for interruptible sleeping IOCTLs 2023-10-26 09:17:40 -07:00
spa_errlog.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
spa_history.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
spa_log_spacemap.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
spa_misc.c BRT: Rework structures and locks to be per-vdev 2024-11-21 08:24:37 -08:00
spa_stats.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
spa.c L2ARC: Stop rebuild before setting spa_final_txg 2024-11-21 08:24:37 -08:00
space_map.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
space_reftree.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
THIRDPARTYLICENSE.cityhash OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
THIRDPARTYLICENSE.cityhash.descrip OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
txg.c vdev probe to slow disk can stall mmp write checker 2024-04-29 14:35:53 -07:00
uberblock.c Provide macros for setting and getting blkptr birth times 2024-03-25 15:01:54 -07:00
unique.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_draid_rand.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_draid.c Always validate checksums for Direct I/O reads 2024-10-09 13:45:06 -07:00
vdev_indirect_births.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
vdev_indirect_mapping.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
vdev_indirect.c Always validate checksums for Direct I/O reads 2024-10-09 13:45:06 -07:00
vdev_initialize.c Add support for parallel pool exports 2024-05-14 08:57:41 -07:00
vdev_label.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
vdev_mirror.c Always validate checksums for Direct I/O reads 2024-10-09 13:45:06 -07:00
vdev_missing.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_queue.c Fix accounting error for pending sync IO ops in zpool iostat 2023-11-07 09:06:14 -08:00
vdev_raidz_math_aarch64_neon_common.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_aarch64_neon.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_aarch64_neonx2.c Fix Clang 15 compilation errors 2022-11-30 13:46:26 -08:00
vdev_raidz_math_avx2.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_avx512bw.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_avx512f.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_impl.h RAIDZ: Use cache blocking during parity math 2023-10-30 14:54:27 -07:00
vdev_raidz_math_powerpc_altivec_common.h Linux ppc64le ieee128 compat: Do not redefine __asm on external headers 2023-01-13 10:58:58 -08:00
vdev_raidz_math_powerpc_altivec.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_scalar.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_sse2.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_ssse3.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math.c Miscellaneous FreBSD compilation bugfixes 2023-04-06 10:35:02 -07:00
vdev_raidz.c Always validate checksums for Direct I/O reads 2024-10-09 13:45:06 -07:00
vdev_rebuild.c Add support for parallel pool exports 2024-05-14 08:57:41 -07:00
vdev_removal.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
vdev_root.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_trim.c Add support for parallel pool exports 2024-05-14 08:57:41 -07:00
vdev.c On the first vdev open ignore impossible ashift hints 2024-11-01 09:49:00 -07:00
zap_leaf.c ZAP: Some cleanups/micro-optimizations 2024-03-21 16:43:53 -07:00
zap_micro.c ZAP: Add by_dnode variants to lookup/prefetch_uint64 2024-11-21 08:24:37 -08:00
zap.c Support for longnames for files/directories (Linux part) 2024-10-01 13:40:27 -07:00
zcp_get.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
zcp_global.c OpenZFS 8600 - ZFS channel programs - snapshot 2018-02-08 15:29:24 -08:00
zcp_iter.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
zcp_set.c Support setting user properties in a channel program 2020-02-14 13:41:42 -08:00
zcp_synctask.c Add zfs.sync.snapshot_rename 2022-09-02 13:31:19 -07:00
zcp.c Simplify issig(). 2024-05-29 10:49:11 -07:00
zfeature.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
zfs_byteswap.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_chksum.c Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
zfs_fm.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
zfs_fuid.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
zfs_impl.c Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
zfs_ioctl.c Support for longnames for files/directories (Linux part) 2024-10-01 13:40:27 -07:00
zfs_log.c zfs_log: add flex array fields to log record structs 2024-09-27 09:18:11 -07:00
zfs_onexit.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
zfs_quota.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
zfs_ratelimit.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_replay.c zfs_log: add flex array fields to log record structs 2024-09-27 09:18:11 -07:00
zfs_rlock.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_sa.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_vnops.c zfs(4): remove "experimental" from zfs_bclone_enabled 2024-11-04 16:46:39 -08:00
zfs_znode.c Support for longnames for files/directories (Linux part) 2024-10-01 13:40:27 -07:00
zil.c zfs: add bounds checking to zil_parse (#16308) 2024-07-31 17:17:04 -07:00
zio_checksum.c Move "no name changes" from compression to checksum table 2024-11-21 08:24:37 -08:00
zio_compress.c Move "no name changes" from compression to checksum table 2024-11-21 08:24:37 -08:00
zio_inject.c vdev probe to slow disk can stall mmp write checker 2024-04-29 14:35:53 -07:00
zio.c zio: Avoid sleeping in the I/O path 2024-11-21 08:24:37 -08:00
zle.c compress: change compression providers API to use ABDs 2024-08-22 16:22:24 -07:00
zrlock.c Micro-optimize zrl_remove() 2022-11-29 09:26:03 -08:00
zthr.c Switch from _Noreturn to __attribute__((noreturn)) 2022-03-23 08:51:00 -07:00
zvol.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00