c/mirror_zfs
1
0
Fork 0
mirror of https://git.proxmox.com/git/mirror_zfs.git synced 2024-11-17 18:11:00 +03:00
Code Issues Packages Projects Releases Wiki Activity
1fbfcf1159
mirror_zfs/include/sys
History
Matthew Ahrens dbeb879699 OpenZFS 8199 - multi-threaded dmu_object_alloc() 
dmu_object_alloc() is single-threaded, so when multiple threads are
creating files in a single filesystem, they spend a lot of time waiting
for the os_obj_lock.  To improve performance of multi-threaded file
creation, we must make dmu_object_alloc() typically not grab any
filesystem-wide locks.

The solution is to have a "next object to allocate" for each CPU. Each
of these "next object"s is in a different block of the dnode object, so
that concurrent allocation holds dnodes in different dbufs.  When a
thread's "next object" reaches the end of a chunk of objects (by default
4 blocks worth -- 128 dnodes), it will be reset to the per-objset
os_obj_next, which will be increased by a chunk of objects (128).  Only
when manipulating the os_obj_next will we need to grab the os_obj_lock.
This decreases lock contention dramatically, because each thread only
needs to grab the os_obj_lock briefly, once per 128 allocations.

This results in a 70% performance improvement to multi-threaded object
creation (where each thread is creating objects in its own directory),
from 67,000/sec to 115,000/sec, with 8 CPUs.

Work sponsored by Intel Corp.

Authored by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Ned Bass <bass6@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>

OpenZFS-issue: https://www.illumos.org/issues/8199
OpenZFS-commit: https://github.com/openzfs/openzfs/pull/374
Closes #4703
Closes #6117
2017-06-09 09:43:26 -07:00
..
crypto OpenZFS 4185 - add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R 2016-10-03 14:51:15 -07:00
fm Additional Information for Zedlets 2017-04-03 14:23:02 -07:00
fs Implemented zpool sync command 2017-05-19 12:33:11 -07:00
sysevent Bring over illumos ZFS FMA logic -- phase 1 2016-09-01 11:39:45 -07:00
abd.h ABD optimized page allocation code 2016-11-29 14:34:33 -08:00
arc_impl.h OpenZFS 7968 - multi-threaded spa_sync() 2017-03-20 18:36:00 -07:00
arc.h zfs_arc_num_sublists_per_state should be common to all multilists 2017-02-15 15:49:33 -08:00
avl_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
avl.h Performance optimization of AVL tree comparator functions 2016-08-31 14:35:34 -07:00
blkptr.h Illumos 4757, 4913 2014-08-01 14:28:05 -07:00
bplist.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
bpobj.h Illumos 5810 - zdb should print details of bpobj 2015-05-11 15:10:24 -07:00
bptree.h Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t 2014-08-06 14:48:41 -07:00
bqueue.h Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
dbuf.h fix: Shift exponent too large 2016-09-29 15:55:41 -07:00
ddt.h DLPX-44812 integrate EP-220 large memory scalability 2016-11-29 14:34:27 -08:00
dmu_impl.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
dmu_objset.h OpenZFS 8199 - multi-threaded dmu_object_alloc() 2017-06-09 09:43:26 -07:00
dmu_send.h DLPX-40252 integrate EP-476 compressed zfs send/receive 2016-09-13 09:58:58 -07:00
dmu_traverse.h OpenZFS 2605, 6980, 6902 2016-06-28 13:47:02 -07:00
dmu_tx.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
dmu_zfetch.h OpenZFS 6322 - ZFS indirect block predictive prefetch 2016-08-30 14:26:55 -07:00
dmu.h OpenZFS 8155 - simplify dmu_write_policy handling of pre-compressed buffers 2017-06-07 14:16:01 -04:00
dnode.h OpenZFS 7968 - multi-threaded spa_sync() 2017-03-20 18:36:00 -07:00
dsl_bookmark.h Illumos 4368, 4369. 2014-07-29 10:55:29 -07:00
dsl_dataset.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
dsl_deadlist.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
dsl_deleg.h Add support for user/group dnode accounting & quota 2016-10-07 09:45:13 -07:00
dsl_destroy.h Illumos #3888 2013-11-04 11:18:14 -08:00
dsl_dir.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
dsl_pool.h OpenZFS 7968 - multi-threaded spa_sync() 2017-03-20 18:36:00 -07:00
dsl_prop.h Illumos 6171 - dsl_prop_unregister() slows down dataset eviction. 2016-01-12 10:53:12 -08:00
dsl_scan.h Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t 2014-08-06 14:48:41 -07:00
dsl_synctask.h Illumos 4951 - ZFS administrative commands should use reserved space 2015-05-04 09:41:10 -07:00
dsl_userhold.h Illumos #3740 2013-11-04 11:17:48 -08:00
edonr.h OpenZFS 4185 - add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R 2016-10-03 14:51:15 -07:00
efi_partition.h Fix spelling 2017-01-03 11:31:18 -06:00
Makefile.am DLPX-44812 integrate EP-220 large memory scalability 2016-11-29 14:34:27 -08:00
metaslab_impl.h OpenZFS 7613 - ms_freetree[4] is only used in syncing context 2017-01-26 15:27:19 -08:00
metaslab.h OpenZFS 7303 - dynamic metaslab selection 2017-01-12 11:52:56 -08:00
mntent.h Make zfs mount according to relatime config in dataset 2016-04-05 18:55:59 -07:00
multilist.h OpenZFS 7968 - multi-threaded spa_sync() 2017-03-20 18:36:00 -07:00
nvpair_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
nvpair.h Replace __va_list with va_list 2014-08-13 10:35:00 -07:00
pathname.h Add pn_alloc()/pn_free() functions 2016-04-21 09:49:25 -07:00
policy.h Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
range_tree.h Illumos #4374 2014-07-30 09:20:35 -07:00
refcount.h Linux 4.11 compat: avoid refcount_t name conflict 2017-02-28 16:10:18 -08:00
rrwlock.h Illumos 5008 - lock contention (rrw_exit) while running a read only load 2015-07-06 09:34:13 -07:00
sa_impl.h Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
sa.h Remove unused sa_update_from_cb() 2016-12-01 16:39:06 -07:00
sdt.h Swap DTRACE_PROBE* with Linux tracepoints 2014-11-17 11:13:55 -08:00
sha2.h OpenZFS 4185 - add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R 2016-10-03 14:51:15 -07:00
skein.h OpenZFS 4185 - add new cryptographic checksums to ZFS: SHA-512, Skein, Edon-R 2016-10-03 14:51:15 -07:00
spa_boot.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
spa_checksum.h Implementation of AVX2 optimized Fletcher-4 2016-06-02 14:30:51 -07:00
spa_impl.h OpenZFS 7743 - per-vdev-zaps init path for upgrade 2017-01-13 13:50:22 -08:00
spa.h More ashift improvements 2017-05-03 09:31:05 -07:00
space_map.h Illumos 5164-5165 - space map fixes 2014-10-23 15:30:32 -07:00
space_reftree.h Illumos #4101, #4102, #4103, #4105, #4106 2014-07-22 09:39:16 -07:00
sysevent.h OpenZFS 5997 - FRU field not set during pool creation and never updated 2016-08-12 13:06:48 -07:00
trace_acl.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_arc.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_common.h OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
trace_dbgmsg.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_dbuf.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_dmu.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
trace_dnode.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_multilist.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_txg.h Fix build-it compilation regression 2017-01-24 08:50:15 -08:00
trace_zil.h OpenZFS 7578 - Fix/improve some aspects of ZIL writing 2017-06-09 09:15:37 -07:00
trace_zio.h Use cstyle -cpP in make cstyle check 2016-12-12 10:46:26 -08:00
trace_zrlock.h Use cstyle -cpP in make cstyle check 2016-12-12 10:46:26 -08:00
trace.h Remove duplicate typedefs from trace.h 2015-01-06 16:53:24 -08:00
txg_impl.h Fix spelling 2017-01-03 11:31:18 -06:00
txg.h OpenZFS 8063 - verify that we do not attempt to access inactive txg 2017-05-10 13:52:22 -04:00
u8_textprep_data.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
u8_textprep.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
uberblock_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
uberblock.h Illumos 5347 - idle pool may run itself out of space 2015-07-14 10:35:21 -07:00
uio_impl.h Add basic uio support 2011-02-10 09:21:43 -08:00
unique.h Illumos #3742 2013-11-04 10:55:25 -08:00
uuid.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
vdev_disk.h Remove custom root pool import code 2016-08-11 11:19:34 -07:00
vdev_file.h Use a dedicated taskq for vdev_file 2016-12-21 10:47:15 -08:00
vdev_impl.h Skip spurious resilver IO on raidz vdev 2017-05-12 17:28:03 -07:00
vdev_raidz_impl.h Use cstyle -cpP in make cstyle check 2016-12-12 10:46:26 -08:00
vdev_raidz.h Use cstyle -cpP in make cstyle check 2016-12-12 10:46:26 -08:00
vdev.h Skip spurious resilver IO on raidz vdev 2017-05-12 17:28:03 -07:00
xvattr.h Fix spelling 2017-01-03 11:31:18 -06:00
zap_impl.h OpenZFS 7793 - ztest fails assertion in dmu_tx_willuse_space 2017-03-07 09:51:59 -08:00
zap_leaf.h Illumos 5056 - ZFS deadlock on db_mtx and dn_holds 2015-04-28 16:25:34 -07:00
zap.h OpenZFS 1300 - filename normalization doesn't work for removes 2017-02-02 14:13:41 -08:00
zfeature.h Revert "zhack: Add 'feature disable' command" 2016-05-17 11:52:07 -07:00
zfs_acl.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zfs_context.h Linux 4.12 compat: PF_FSTRANS was removed 2017-05-09 10:38:46 -07:00
zfs_ctldir.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zfs_debug.h OpenZFS 7277 - zdb should be able to print zfs_dbgmsg's 2017-01-28 12:16:43 -08:00
zfs_delay.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
zfs_dir.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zfs_fuid.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zfs_ioctl.h Fix large dnode send stream flag conflict 2017-05-18 10:02:16 -07:00
zfs_onexit.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
zfs_ratelimit.h Add missing *_destroy/*_fini calls 2017-05-04 19:26:28 -04:00
zfs_rlock.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zfs_sa.h Illumos 5027 - zfs large block support 2015-05-11 12:23:16 -07:00
zfs_stat.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
zfs_vfsops.h Linux 4.12 compat: super_setup_bdi_name() 2017-05-02 09:46:18 -07:00
zfs_vnops.h Rename zfs_* functions 2017-03-10 09:51:35 -08:00
zfs_znode.h Rename zfs_sb_t -> zfsvfs_t 2017-03-10 09:51:33 -08:00
zil_impl.h OpenZFS 7578 - Fix/improve some aspects of ZIL writing 2017-06-09 09:15:37 -07:00
zil.h OpenZFS 7578 - Fix/improve some aspects of ZIL writing 2017-06-09 09:15:37 -07:00
zio_checksum.h Remove dependency on linear ABD 2017-03-29 12:24:51 -07:00
zio_compress.h DLPX-44812 integrate EP-220 large memory scalability 2016-11-29 14:34:27 -08:00
zio_impl.h Compile zio.h and zio_impl.h mutual include 2016-12-01 16:36:25 -07:00
zio_priority.h Add -lhHpw options to "zpool iostat" for avg latency, histograms, & queues 2016-05-12 12:36:32 -07:00
zio.h OpenZFS 7578 - Fix/improve some aspects of ZIL writing 2017-06-09 09:15:37 -07:00
zpl.h Use cstyle -cpP in make cstyle check 2016-12-12 10:46:26 -08:00
zrlock.h OpenZFS 6328 - Fix cstyle errors in zfs codebase 2017-01-12 09:42:11 -08:00
zvol.h Suspend/resume zvol for recv and rollback 2017-01-19 13:56:36 -08:00