mirror_zfs/include/sys
Alexander Motin 32c0b6468c Optimize allocation throttling
Remove mc_lock use from metaslab_class_throttle_*().  The math there
is based on refcounts and so atomic, so the only race possible there
is between zfs_refcount_count() and zfs_refcount_add().  But in most
cases metaslab_class_throttle_reserve() is called with the allocator
lock held, which covers the race.  In cases where the lock is not
held, GANG_ALLOCATION() or METASLAB_MUST_RESERVE are set, and so we
do not use zfs_refcount_count().  And even if we assume some other
non-existing scenario, the worst that may happen from this race is
few more I/Os get to allocation earlier, that is not a problem.

Move locks and data of different allocators into different cache
lines to avoid false sharing.  Group spa_alloc_* arrays together
into single array of aligned struct spa_alloc spa_allocs.  Align
struct metaslab_class_allocator.

Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Don Brady <don.brady@delphix.com>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Closes #12314
2021-09-14 12:40:15 -07:00
..
crypto gcc 11 cleanup 2021-06-24 13:13:40 -07:00
fm file reference counts can get corrupted 2021-09-14 12:37:38 -07:00
fs Add "compatibility" property for zpool feature sets 2021-02-17 21:30:45 -08:00
lua FreeBSD: Reduce stack usage of Lua 2020-09-22 16:03:11 -07:00
sysevent Avoid installing kernel headers on FreeBSD 2020-06-27 17:40:14 -07:00
zstd zstd: track allocator statistics 2020-10-30 15:26:10 -07:00
abd_impl.h Fix abd leak, kmem_free correct size of abd_t 2021-09-14 12:22:28 -07:00
abd.h FreeBSD: Hardcode abd_chunk_size to PAGE_SIZE 2021-09-14 12:36:44 -07:00
aggsum.h More aggsum optimizations 2021-06-09 13:05:34 -07:00
arc_impl.h Minor ARC optimizations 2021-09-14 12:39:48 -07:00
arc.h Minor ARC optimizations 2021-09-14 12:39:48 -07:00
avl_impl.h
avl.h Restore avl_update() calls and related functions 2020-06-03 09:49:32 -07:00
bitops.h Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
blkptr.h
bplist.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
bpobj.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
bptree.h
bqueue.h
btree.h Fix typos 2020-06-09 21:24:09 -07:00
dataset_kstats.h Introduce write-mostly sums 2021-06-09 13:05:34 -07:00
dbuf.h dprintf_dnode: strcpy -> strlcpy 2021-09-14 12:37:38 -07:00
ddt.h Appease GCC sprintf warnings found on Fedora 32/GCC 10.0.1 2020-08-24 10:32:59 -07:00
dmu_impl.h Remove UIO_ZEROCOPY functions structures 2020-10-30 10:00:33 -07:00
dmu_objset.h Re-embed multilist_t storage 2021-06-10 10:50:16 -07:00
dmu_recv.h Removed duplicated includes 2021-03-22 12:34:58 -07:00
dmu_redact.h Suppress cppcheck invalidSyntax warninigs 2021-03-05 17:56:35 -08:00
dmu_send.h Removed duplicated includes 2021-03-22 12:34:58 -07:00
dmu_traverse.h
dmu_tx.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
dmu_zfetch.h Split dmu_zfetch() speculation and execution parts 2021-03-19 22:56:11 -07:00
dmu.h Set aside a metaslab for ZIL blocks 2021-01-21 15:12:54 -08:00
dnode.h dprintf_dnode: strcpy -> strlcpy 2021-09-14 12:37:38 -07:00
dsl_bookmark.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
dsl_crypt.h dmu_objset_from_ds must be called with dp_config_rwlock held 2020-03-12 10:55:02 -07:00
dsl_dataset.h implicit conversion from 'boolean_t' to 'ds_hold_flags_t' 2020-12-27 16:31:02 -08:00
dsl_deadlist.h Add fast path for zfs_ioc_space_snaps() handling of empty_bpobj 2019-08-20 11:34:52 -07:00
dsl_deleg.h
dsl_destroy.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
dsl_dir.h Introduce dsl_dir_diduse_transfer_space() 2021-09-14 12:38:51 -07:00
dsl_pool.h Eliminate Linux specific inode usage from common code 2019-12-11 11:53:57 -08:00
dsl_prop.h Support inheriting properties in channel programs 2020-01-22 17:03:17 -08:00
dsl_scan.h Use dsl_scan_setup_check() to setup a scrub 2021-04-14 13:19:49 -07:00
dsl_synctask.h Add upper bound for slop space calculation 2021-02-24 09:52:43 -08:00
dsl_userhold.h
edonr.h
efi_partition.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
frame.h Linux 5.10 compat: frame.h renamed objtool.h 2020-11-02 22:01:10 +00:00
hkdf.h
Makefile.am Restore FreeBSD resource usage accounting 2021-02-19 22:34:33 -08:00
metaslab_impl.h Optimize allocation throttling 2021-09-14 12:40:15 -07:00
metaslab.h Only examine best metaslabs on each vdev 2020-12-16 14:40:05 -08:00
mmp.h Add zfs_multihost_interval tunable handler for FreeBSD 2020-06-23 13:32:42 -07:00
mntent.h Add FreeBSD required defines to mntent.h 2019-11-30 15:49:09 -08:00
mod.h Replace ZFS on Linux references with OpenZFS 2020-10-08 20:10:13 -07:00
multilist.h Re-embed multilist_t storage 2021-06-10 10:50:16 -07:00
note.h
nvpair_impl.h
nvpair.h FreeBSD: make adjustments for the standalone environment 2020-10-13 21:05:49 -07:00
objlist.h
pathname.h Replace ZFS on Linux references with OpenZFS 2020-10-08 20:10:13 -07:00
qat.h QAT related bug fixes 2019-09-12 13:33:44 -07:00
range_tree.h Improve compatibility with C++ consumers 2020-06-06 12:54:04 -07:00
rrwlock.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
sa_impl.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
sa.h Extending FreeBSD UIO Struct 2021-01-20 21:27:30 -08:00
skein.h
spa_boot.h
spa_checkpoint.h
spa_checksum.h
spa_impl.h Optimize allocation throttling 2021-09-14 12:40:15 -07:00
spa_log_spacemap.h
spa.h Optimize small random numbers generation 2021-09-14 12:10:17 -07:00
space_map.h Extend zdb to print inconsistencies in livelists and metaslabs 2020-07-14 17:51:05 -07:00
space_reftree.h Reduce loaded range tree memory usage 2019-10-09 10:36:03 -07:00
sysevent.h
txg_impl.h Fix typos in include/ 2019-08-30 09:53:15 -07:00
txg.h Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
u8_textprep_data.h
u8_textprep.h Throw const on some strings 2020-10-02 17:44:10 -07:00
uberblock_impl.h
uberblock.h
uio_impl.h Cleaning up uio headers 2021-02-20 20:16:50 -08:00
unique.h
uuid.h
vdev_disk.h Make struct vdev_disk_t be platform private 2020-06-16 11:43:33 -07:00
vdev_draid.h Fix various typos 2021-04-07 13:27:11 -07:00
vdev_file.h Add zfs_file_* interface, remove vnodes 2019-11-21 09:32:57 -08:00
vdev_impl.h Ratelimit deadman zevents as with delay zevents 2021-04-14 13:19:49 -07:00
vdev_indirect_births.h
vdev_indirect_mapping.h
vdev_initialize.h
vdev_raidz_impl.h Fix dRAID sequential resilver silent damage handling 2021-05-27 22:31:56 -07:00
vdev_raidz.h Clean up RAIDZ/DRAID ereport code 2021-03-19 16:22:10 -07:00
vdev_rebuild.h Fix various typos 2021-04-07 13:27:11 -07:00
vdev_removal.h
vdev_trim.h Trim L2ARC 2020-06-09 10:15:08 -07:00
vdev.h Use a helper function to clarify gang block size 2021-03-26 11:19:35 -07:00
xvattr.h
zap_impl.h
zap_leaf.h
zap.h
zcp_global.h
zcp_iter.h
zcp_prop.h
zcp_set.h Support setting user properties in a channel program 2020-02-14 13:41:42 -08:00
zcp.h filesystem_limit/snapshot_limit is incorrectly enforced against root 2020-07-11 17:18:02 -07:00
zfeature.h
zfs_acl.h Return an error code from zfs_acl_chmod_setattr 2019-11-01 10:19:11 -07:00
zfs_bootenv.h zfs label bootenv should store data as nvlist 2020-09-15 15:42:27 -07:00
zfs_context.h Optimize small random numbers generation 2021-09-14 12:10:17 -07:00
zfs_debug.h Annotated dprintf as printf-like 2021-06-24 13:12:36 -07:00
zfs_delay.h
zfs_file.h file reference counts can get corrupted 2021-09-14 12:37:38 -07:00
zfs_fuid.h Replace sprintf()->snprintf() and strcpy()->strlcpy() 2020-06-07 11:42:12 -07:00
zfs_ioctl_impl.h libzfs: On FreeBSD, use MNT_NOWAIT with getfsstat 2021-06-09 13:05:34 -07:00
zfs_ioctl.h file reference counts can get corrupted 2021-09-14 12:37:38 -07:00
zfs_onexit.h file reference counts can get corrupted 2021-09-14 12:37:38 -07:00
zfs_project.h Minor diff reduction with ZoF in include/sys 2019-11-27 11:11:03 -08:00
zfs_quota.h File incorrectly zeroed when receiving incremental stream that toggles -L 2020-06-09 10:41:01 -07:00
zfs_racct.h Restore FreeBSD resource usage accounting 2021-02-19 22:34:33 -08:00
zfs_ratelimit.h
zfs_refcount.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
zfs_rlock.h Add a "try" operation for range locks 2020-07-06 11:53:31 -07:00
zfs_sa.h Extending FreeBSD UIO Struct 2021-01-20 21:27:30 -08:00
zfs_stat.h
zfs_sysfs.h
zfs_vfsops.h Add 'zfs rename -u' to rename without remounting 2020-09-01 16:14:16 -07:00
zfs_vnops.h Extending FreeBSD UIO Struct 2021-01-20 21:27:30 -08:00
zfs_znode.h Rename zfs_inode_update to zfs_znode_update_vfs 2021-02-09 11:17:29 -08:00
zil_impl.h
zil.h Fix zfs_get_data access to files with wrong generation 2021-03-19 22:53:31 -07:00
zio_checksum.h
zio_compress.h Add zstd support to zfs 2020-08-20 10:30:06 -07:00
zio_crypt.h Rename refcount.h to zfs_refcount.h 2020-07-29 16:35:33 -07:00
zio_impl.h Add zstd support to zfs 2020-08-20 10:30:06 -07:00
zio_priority.h Add device rebuild feature 2020-07-03 11:05:50 -07:00
zio.h A few fixes of callback typecasting (for the upcoming ClangCFI) 2021-09-14 12:39:48 -07:00
zrlock.h
zthr.h Introduce names for ZTHRs 2020-07-29 09:43:33 -07:00
zvol_impl.h Fix zfs_get_data access to files with wrong generation 2021-03-19 22:53:31 -07:00
zvol.h async zvol minor node creation interferes with receive 2020-02-03 09:33:14 -08:00