c/mirror_zfs
1
0
Fork 0
mirror of https://git.proxmox.com/git/mirror_zfs.git synced 2024-11-17 10:01:01 +03:00
Code Issues Packages Projects Releases Wiki Activity
68cbd56e18
mirror_zfs/include/sys
History
Ned Bass 68cbd56e18 Backfill metadnode more intelligently 
Only attempt to backfill lower metadnode object numbers if at least
4096 objects have been freed since the last rescan, and at most once
per transaction group. This avoids a pathology in dmu_object_alloc()
that caused O(N^2) behavior for create-heavy workloads and
substantially improves object creation rates.  As summarized by
@mahrens in #4636:

"Normally, the object allocator simply checks to see if the next
object is available. The slow calls happened when dmu_object_alloc()
checks to see if it can backfill lower object numbers. This happens
every time we move on to a new L1 indirect block (i.e. every 32 *
128 = 4096 objects).  When re-checking lower object numbers, we use
the on-disk fill count (blkptr_t:blk_fill) to quickly skip over
indirect blocks that don’t have enough free dnodes (defined as an L2
with at least 393,216 of 524,288 dnodes free). Therefore, we may
find that a block of dnodes has a low (or zero) fill count, and yet
we can’t allocate any of its dnodes, because they've been allocated
in memory but not yet written to disk. In this case we have to hold
each of the dnodes and then notice that it has been allocated in
memory.

The end result is that allocating N objects in the same TXG can
require CPU usage proportional to N^2."

Add a tunable dmu_rescan_dnode_threshold to define the number of
objects that must be freed before a rescan is performed. Don't bother
to export this as a module option because testing doesn't show a
compelling reason to change it. The vast majority of the performance
gain comes from limit the rescan to at most once per TXG.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2016-06-24 13:13:12 -07:00
..
fm Illumos 3749 - zfs event processing should work on R/O root filesystems 2016-01-12 14:42:32 -08:00
fs Add request size histograms (-r) to zpool iostat, minor man page fix 2016-05-25 15:49:35 -07:00
arc_impl.h OpenZFS 6513 - partially filled holes lose birth time 2016-06-21 10:55:13 -07:00
arc.h OpenZFS 6513 - partially filled holes lose birth time 2016-06-21 10:55:13 -07:00
avl_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
avl.h Illumos 4745 - fix AVL code misspellings 2015-07-10 11:58:37 -07:00
blkptr.h Illumos 4757, 4913 2014-08-01 14:28:05 -07:00
bplist.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
bpobj.h Illumos 5810 - zdb should print details of bpobj 2015-05-11 15:10:24 -07:00
bptree.h Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t 2014-08-06 14:48:41 -07:00
bqueue.h Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
dbuf.h Illumos 6844 - dnode_next_offset can detect fictional holes 2016-04-27 16:24:15 -07:00
ddt.h Add ddt, ddt_entry, and l2arc_hdr caches 2014-01-07 10:33:11 -08:00
dmu_impl.h Illumos 4757, 4913 2014-08-01 14:28:05 -07:00
dmu_objset.h Backfill metadnode more intelligently 2016-06-24 13:13:12 -07:00
dmu_send.h Illumos 5765 - add support for estimating send stream size with lzc_send_space when source is a bookmark 2015-05-13 09:03:59 -07:00
dmu_traverse.h Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t 2014-08-06 14:48:41 -07:00
dmu_tx.h dmu_tx kstat cleanup 2014-03-04 12:22:24 -08:00
dmu_zfetch.h Illumos 5987 - zfs prefetch code needs work 2016-01-12 09:02:33 -08:00
dmu.h OpenZFS 6736 - ZFS per-vdev ZAPs 2016-05-02 14:27:45 -07:00
dnode.h Illumos 5141 - zfs minimum indirect block size is 4K 2016-01-12 14:11:31 -08:00
dsl_bookmark.h Illumos 4368, 4369. 2014-07-29 10:55:29 -07:00
dsl_dataset.h Illumos 6171 - dsl_prop_unregister() slows down dataset eviction. 2016-01-12 10:53:12 -08:00
dsl_deadlist.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
dsl_deleg.h Illumos 4368, 4369. 2014-07-29 10:55:29 -07:00
dsl_destroy.h Illumos #3888 2013-11-04 11:18:14 -08:00
dsl_dir.h Illumos 6171 - dsl_prop_unregister() slows down dataset eviction. 2016-01-12 10:53:12 -08:00
dsl_pool.h Illumos 5981 - Deadlock in dmu_objset_find_dp 2015-07-06 09:31:35 -07:00
dsl_prop.h Illumos 6171 - dsl_prop_unregister() slows down dataset eviction. 2016-01-12 10:53:12 -08:00
dsl_scan.h Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t 2014-08-06 14:48:41 -07:00
dsl_synctask.h Illumos 4951 - ZFS administrative commands should use reserved space 2015-05-04 09:41:10 -07:00
dsl_userhold.h Illumos #3740 2013-11-04 11:17:48 -08:00
efi_partition.h Ext4's typical GPT partition type not recognized 2015-12-04 09:27:00 -08:00
Makefile.am SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
metaslab_impl.h Remove fastwrite mutex 2016-01-15 15:38:35 -08:00
metaslab.h Illumos 5213 - panic in metaslab_init due to space_map_open returning ENXIO 2014-11-14 15:37:45 -08:00
mntent.h Make zfs mount according to relatime config in dataset 2016-04-05 18:55:59 -07:00
multilist.h Illumos 5497 - lock contention on arcs_mtx 2015-06-11 10:27:25 -07:00
nvpair_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
nvpair.h Replace __va_list with va_list 2014-08-13 10:35:00 -07:00
pathname.h Add pn_alloc()/pn_free() functions 2016-04-21 09:49:25 -07:00
policy.h Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
range_tree.h Illumos #4374 2014-07-30 09:20:35 -07:00
refcount.h Illumos 5045 - use atomic_{inc,dec}_* instead of atomic_add_* 2016-01-15 15:38:36 -08:00
rrwlock.h Illumos 5008 - lock contention (rrw_exit) while running a read only load 2015-07-06 09:34:13 -07:00
sa_impl.h Illumos 5056 - ZFS deadlock on db_mtx and dn_holds 2015-04-28 16:25:34 -07:00
sa.h Prevent SA length overflow 2015-12-30 13:20:12 -08:00
sdt.h Swap DTRACE_PROBE* with Linux tracepoints 2014-11-17 11:13:55 -08:00
spa_boot.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
spa_checksum.h Implementation of AVX2 optimized Fletcher-4 2016-06-02 14:30:51 -07:00
spa_impl.h OpenZFS 6736 - ZFS per-vdev ZAPs 2016-05-02 14:27:45 -07:00
spa.h Implementation of AVX2 optimized Fletcher-4 2016-06-02 14:30:51 -07:00
space_map.h Illumos 5164-5165 - space map fixes 2014-10-23 15:30:32 -07:00
space_reftree.h Illumos #4101, #4102, #4103, #4105, #4106 2014-07-22 09:39:16 -07:00
trace_acl.h Kill znode->z_gen field 2016-05-19 13:06:14 -07:00
trace_arc.h OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
trace_common.h OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
trace_dbgmsg.h SET_ERROR should print strings 2016-01-15 15:38:35 -08:00
trace_dbuf.h SET_ERROR should print strings 2016-01-15 15:38:35 -08:00
trace_dmu.h Fix build failure with Linux 4.1 and FTRACE 2015-07-29 07:35:06 -07:00
trace_dnode.h Fix build failure with Linux 4.1 and FTRACE 2015-07-29 07:35:06 -07:00
trace_multilist.h Fix build failure with Linux 4.1 and FTRACE 2015-08-18 16:47:21 -07:00
trace_txg.h Fix build failure with Linux 4.1 and FTRACE 2015-07-29 07:35:06 -07:00
trace_zil.h Fix build failure with Linux 4.1 and FTRACE 2015-07-29 07:35:06 -07:00
trace_zio.h OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
trace_zrlock.h SET_ERROR should print strings 2016-01-15 15:38:35 -08:00
trace.h Remove duplicate typedefs from trace.h 2015-01-06 16:53:24 -08:00
txg_impl.h Illumos #4045 write throttle & i/o scheduler performance work 2013-12-06 09:32:43 -08:00
txg.h Illumos 4753 - increase number of outstanding async writes when sync task is waiting 2014-09-23 13:50:55 -07:00
u8_textprep_data.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
u8_textprep.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
uberblock_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
uberblock.h Illumos 5347 - idle pool may run itself out of space 2015-07-14 10:35:21 -07:00
uio_impl.h Add basic uio support 2011-02-10 09:21:43 -08:00
unique.h Illumos #3742 2013-11-04 10:55:25 -08:00
uuid.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
vdev_disk.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
vdev_file.h Update all default taskq settings 2015-06-25 08:58:16 -07:00
vdev_impl.h Add -lhHpw options to "zpool iostat" for avg latency, histograms, & queues 2016-05-12 12:36:32 -07:00
vdev_raidz_impl.h SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
vdev_raidz.h SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
vdev.h Add -lhHpw options to "zpool iostat" for avg latency, histograms, & queues 2016-05-12 12:36:32 -07:00
xvattr.h Add xvattr support 2011-03-02 11:43:50 -08:00
zap_impl.h Illumos 5027 - zfs large block support 2015-05-11 12:23:16 -07:00
zap_leaf.h Illumos 5056 - ZFS deadlock on db_mtx and dn_holds 2015-04-28 16:25:34 -07:00
zap.h Add zap_prefetch() interface 2015-12-04 09:39:20 -08:00
zfeature.h Revert "zhack: Add 'feature disable' command" 2016-05-17 11:52:07 -07:00
zfs_acl.h Illumos #3742 2013-11-04 10:55:25 -08:00
zfs_context.h Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
zfs_ctldir.h Use spa as key besides objsetid for snapentry 2015-12-08 16:38:56 -08:00
zfs_debug.h Add dbgmsg kstat 2015-09-04 16:08:14 -07:00
zfs_delay.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
zfs_dir.h Prototype/structure update for Linux 2011-02-10 09:27:21 -08:00
zfs_fuid.h Prototype/structure update for Linux 2011-02-10 09:27:21 -08:00
zfs_ioctl.h OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
zfs_onexit.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
zfs_rlock.h Use zfs range locks in ztest 2016-05-17 10:40:30 -07:00
zfs_sa.h Illumos 5027 - zfs large block support 2015-05-11 12:23:16 -07:00
zfs_stat.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
zfs_vfsops.h Fix zsb->z_hold_mtx deadlock 2016-01-15 15:33:45 -08:00
zfs_vnops.h Add pn_alloc()/pn_free() functions 2016-04-21 09:49:25 -07:00
zfs_znode.h Kill znode->z_gen field 2016-05-19 13:06:14 -07:00
zil_impl.h Illumos 5027 - zfs large block support 2015-05-11 12:23:16 -07:00
zil.h Illumos 5269 - zpool import slow 2015-06-09 13:48:02 -07:00
zio_checksum.h Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
zio_compress.h Illumos #3742 2013-11-04 10:55:25 -08:00
zio_impl.h Illumos #3836 2013-11-05 12:14:56 -08:00
zio_priority.h Add -lhHpw options to "zpool iostat" for avg latency, histograms, & queues 2016-05-12 12:36:32 -07:00
zio.h OpenZFS 6513 - partially filled holes lose birth time 2016-06-21 10:55:13 -07:00
zpl.h Linux 4.7 compat: use iterate_shared for concurrent readdir 2016-05-20 11:09:16 -07:00
zrlock.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
zvol.h Add support for asynchronous zvol minor operations 2016-03-10 09:49:22 -08:00