Alexander Motin aef452f108 Improve zfs_blkptr_verify() ...

- Skip config lock enter/exit for embedded blocks.  They have no
DVAs, so there is nothing to check under the lock.
 - Skip CHECKSUM check and properly check PSIZE for embedded blocks.
 - Add static branch predictions for unlikely conditions.
 - Do not verify DVAs for blocks already in ARC.  ARC hit already
"verified" the first (often the only) DVA, and it does not worth to
enter/exit config lock for nothing.

Some profiles show me up to 3% of CPU saving from this change.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Allan Jude <allan@klarasystems.com>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored by:	iXsystems, Inc.
Closes #16387

2024-08-08 15:25:10 -07:00

..

abd.c

abd: add page iterator

2024-03-25 16:50:35 -07:00

aggsum.c

Remove bcopy(), bzero(), bcmp()

2022-03-15 15:13:42 -07:00

arc.c

Improve zfs_blkptr_verify()

2024-08-08 15:25:10 -07:00

blake3_zfs.c

blake3: fix up bogus checksums in face of cpu migration

2023-05-01 17:21:27 -07:00

blkptr.c

Remove bcopy(), bzero(), bcmp()

2022-03-15 15:13:42 -07:00

bplist.c

Use list_remove_head() where possible.

2023-06-09 10:12:52 -07:00

bpobj.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

bptree.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

bqueue.c

Batch enqueue/dequeue for bqueue

2023-01-10 13:39:22 -08:00

brt.c

BRT: Skip getting length in brt_entry_lookup()

2024-03-25 17:13:45 -07:00

btree.c

Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN.

2024-05-10 08:47:21 -07:00

dataset_kstats.c

Update the kstat dataset_name when renaming a zvol

2023-11-07 11:34:50 -08:00

dbuf_stats.c

Consider dnode_t allocations in dbuf cache size accounting

2023-11-17 13:25:53 -08:00

dbuf.c

Block cloning conditionally destroy ARC buffer

2024-08-01 18:22:43 -07:00

ddt_stats.c

ddt: add support for prefetching tables into the ARC

2024-07-26 09:16:18 -07:00

ddt_zap.c

ddt: add support for prefetching tables into the ARC

2024-07-26 09:16:18 -07:00

ddt.c

ddt: add support for prefetching tables into the ARC

2024-07-26 09:16:18 -07:00

dmu_diff.c

Simplify issig().

2024-05-29 10:49:11 -07:00

dmu_object.c

Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN.

2024-05-10 08:47:21 -07:00

dmu_objset.c

Simplify issig().

2024-05-29 10:49:11 -07:00

dmu_recv.c

Simplify issig().

2024-05-29 10:49:11 -07:00

dmu_redact.c

Simplify issig().

2024-05-29 10:49:11 -07:00

dmu_send.c

Simplify issig().

2024-05-29 10:49:11 -07:00

dmu_traverse.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

dmu_tx.c

Cleanup DB_DNODE() macros usage

2024-07-29 14:47:01 -07:00

dmu_zfetch.c

Small fix to prefetch ranges aggregation

2024-04-09 16:14:04 -07:00

dmu.c

dnode: allow storage class to be overridden by object type

2024-07-29 17:05:41 -07:00

dnode_sync.c

For db_marker inherit the db pointer for AVL comparision.

2023-12-11 14:42:06 -08:00

dnode.c

Linux: Report reclaimable memory to kernel as such (#16385)

2024-07-30 11:40:47 -07:00

dsl_bookmark.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

dsl_crypt.c

Allow block cloning across encrypted datasets

2023-12-05 11:03:48 -08:00

dsl_dataset.c

head_errlog: fix use-after-free

2024-07-15 09:05:42 -07:00

dsl_deadlist.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

dsl_deleg.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

dsl_destroy.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

dsl_dir.c

Fix ENOSPC for extended quota

2023-09-28 14:10:07 -07:00

dsl_pool.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

dsl_prop.c

nvpair: Constify string functions

2023-03-14 15:25:50 -07:00

dsl_scan.c

Fix scn_queue races on very old pools

2024-05-09 07:32:59 -07:00

dsl_synctask.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

dsl_userhold.c

nvpair: Constify string functions

2023-03-14 15:25:50 -07:00

edonr_zfs.c

Remove unused Edon-R variants

2023-03-14 15:59:58 -07:00

fm.c

Use list_remove_head() where possible.

2023-06-09 10:12:52 -07:00

gzip.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

hkdf.c

Remove bcopy(), bzero(), bcmp()

2022-03-15 15:13:42 -07:00

lz4_zfs.c

Linux: Report reclaimable memory to kernel as such (#16385)

2024-07-30 11:40:47 -07:00

lz4.c

lz4: Cherrypick fix for CVE-2021-3520

2022-01-12 16:14:36 -08:00

lzjb.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

metaslab.c

Some improvements to metaslabs eviction

2024-05-29 08:53:31 -07:00

mmp.c

vdev probe to slow disk can stall mmp write checker

2024-04-29 14:35:53 -07:00

multilist.c

L2ARC: Relax locking during write

2024-04-09 16:23:19 -07:00

objlist.c

Implement Redacted Send/Receive

2019-06-19 09:48:12 -07:00

pathname.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

range_tree.c

btree: Implement faster binary search algorithm

2023-05-26 10:03:12 -07:00

refcount.c

Switch refcount tracking from lists to AVL-trees.

2023-06-14 08:02:27 -07:00

rrwlock.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

sa.c

Fix sa.c to build on FreeBSD again. (#16403)

2024-08-01 13:04:08 -07:00

sha2_zfs.c

Add generic implementation handling and SHA2 impl

2023-03-02 13:52:21 -08:00

skein_zfs.c

Remove bcopy(), bzero(), bcmp()

2022-03-15 15:13:42 -07:00

spa_checkpoint.c

RAID-Z expansion feature

2023-11-08 10:19:41 -08:00

spa_config.c

Add mutex_enter_interruptible() for interruptible sleeping IOCTLs

2023-10-26 09:17:40 -07:00

spa_errlog.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

spa_history.c

nvpair: Constify string functions

2023-03-14 15:25:50 -07:00

spa_log_spacemap.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

spa_misc.c

dnode: allow storage class to be overridden by object type

2024-07-29 17:05:41 -07:00

spa_stats.c

Cleanup: Specify unsignedness on things that should not be signed

2022-09-27 16:42:41 -07:00

spa.c

Sync AUX label during pool import

2024-08-08 15:16:46 -07:00

space_map.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

space_reftree.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

THIRDPARTYLICENSE.cityhash

OpenZFS 8484 - Implement aggregate sum and use for arc counters

2018-06-06 09:35:59 -07:00

THIRDPARTYLICENSE.cityhash.descrip

OpenZFS 8484 - Implement aggregate sum and use for arc counters

2018-06-06 09:35:59 -07:00

txg.c

vdev probe to slow disk can stall mmp write checker

2024-04-29 14:35:53 -07:00

uberblock.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

unique.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_draid_rand.c

Distributed Spare (dRAID) Feature

2020-11-13 13:51:51 -08:00

vdev_draid.c

zio: rename ZIO_TYPE_IOCTL to ZIO_TYPE_FLUSH

2024-04-11 17:17:23 -07:00

vdev_indirect_births.c

Remove bcopy(), bzero(), bcmp()

2022-03-15 15:13:42 -07:00

vdev_indirect_mapping.c

Remove bcopy(), bzero(), bcmp()

2022-03-15 15:13:42 -07:00

vdev_indirect.c

Don't emit cksum_{actual_expected} in ereport.fs.zfs.checksum events

2023-07-21 11:49:26 -07:00

vdev_initialize.c

Add support for parallel pool exports

2024-05-14 08:57:41 -07:00

vdev_label.c

Sync AUX label during pool import

2024-08-08 15:16:46 -07:00

vdev_mirror.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

vdev_missing.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_queue.c

Fix accounting error for pending sync IO ops in zpool iostat

2023-11-07 09:06:14 -08:00

vdev_raidz_math_aarch64_neon_common.h

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_raidz_math_aarch64_neon.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_raidz_math_aarch64_neonx2.c

Fix Clang 15 compilation errors

2022-11-30 13:46:26 -08:00

vdev_raidz_math_avx2.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_raidz_math_avx512bw.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_raidz_math_avx512f.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_raidz_math_impl.h

RAIDZ: Use cache blocking during parity math

2023-10-30 14:54:27 -07:00

vdev_raidz_math_powerpc_altivec_common.h

Linux ppc64le ieee128 compat: Do not redefine __asm on external headers

2023-01-13 10:58:58 -08:00

vdev_raidz_math_powerpc_altivec.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_raidz_math_scalar.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_raidz_math_sse2.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_raidz_math_ssse3.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_raidz_math.c

Miscellaneous FreBSD compilation bugfixes

2023-04-06 10:35:02 -07:00

vdev_raidz.c

Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN.

2024-05-10 08:47:21 -07:00

vdev_rebuild.c

Add support for parallel pool exports

2024-05-14 08:57:41 -07:00

vdev_removal.c

nvpair: Constify string functions

2023-03-14 15:25:50 -07:00

vdev_root.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

vdev_trim.c

Add support for parallel pool exports

2024-05-14 08:57:41 -07:00

vdev.c

Add some missing vdev properties (#16346)

2024-07-23 16:34:09 -07:00

zap_leaf.c

ZAP: Some cleanups/micro-optimizations

2024-03-21 16:43:53 -07:00

zap_micro.c

ddt: add support for prefetching tables into the ARC

2024-07-26 09:16:18 -07:00

zap.c

zap: reuse zap_leaf_t on dbuf reuse after shrink

2024-05-24 18:55:47 -07:00

zcp_get.c

nvpair: Constify string functions

2023-03-14 15:25:50 -07:00

zcp_global.c

OpenZFS 8600 - ZFS channel programs - snapshot

2018-02-08 15:29:24 -08:00

zcp_iter.c

nvpair: Constify string functions

2023-03-14 15:25:50 -07:00

zcp_set.c

Support setting user properties in a channel program

2020-02-14 13:41:42 -08:00

zcp_synctask.c

Add zfs.sync.snapshot_rename

2022-09-02 13:31:19 -07:00

zcp.c

Simplify issig().

2024-05-29 10:49:11 -07:00

zfeature.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

zfs_byteswap.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

zfs_chksum.c

Add generic implementation handling and SHA2 impl

2023-03-02 13:52:21 -08:00

zfs_fm.c

zio: remove io_cmd and DKIOCFLUSHWRITECACHE

2024-04-11 17:17:11 -07:00

zfs_fuid.c

Use list_remove_head() where possible.

2023-06-09 10:12:52 -07:00

zfs_impl.c

Add generic implementation handling and SHA2 impl

2023-03-02 13:52:21 -08:00

zfs_ioctl.c

ddt: add support for prefetching tables into the ARC

2024-07-26 09:16:18 -07:00

zfs_log.c

Cleanup DB_DNODE() macros usage

2024-07-29 14:47:01 -07:00

zfs_onexit.c

Use list_remove_head() where possible.

2023-06-09 10:12:52 -07:00

zfs_quota.c

Revert "Do not persist user/group/project quota zap objects when unneeded"

2023-10-23 09:55:36 -07:00

zfs_ratelimit.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

zfs_replay.c

FreeBSD: remove support for FreeBSD < 13.0-RELEASE (#16372)

2024-08-05 16:56:45 -07:00

zfs_rlock.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

zfs_sa.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

zfs_vnops.c

Simplify issig().

2024-05-29 10:49:11 -07:00

zil.c

zfs: add bounds checking to zil_parse (#16308)

2024-07-31 17:17:04 -07:00

zio_checksum.c

Provide macros for setting and getting blkptr birth times

2024-03-25 15:01:54 -07:00

zio_compress.c

Skip memory allocation when compressing holes

2023-02-27 14:41:02 -08:00

zio_inject.c

vdev probe to slow disk can stall mmp write checker

2024-04-29 14:35:53 -07:00

zio.c

Improve zfs_blkptr_verify()

2024-08-08 15:25:10 -07:00

zle.c

Replace dead opensolaris.org license link

2022-07-11 14:16:13 -07:00

zrlock.c

Micro-optimize zrl_remove()

2022-11-29 09:26:03 -08:00

zthr.c

Switch from _Noreturn to __attribute__((noreturn))

2022-03-23 08:51:00 -07:00

zvol.c

zvol: ensure device minors are properly cleaned up

2024-08-06 12:08:14 -07:00