c/mirror_zfs
1
0
Fork 0
mirror of https://git.proxmox.com/git/mirror_zfs.git synced 2024-11-17 18:11:00 +03:00
Code Issues Packages Projects Releases Wiki Activity
938cfeb0f2
mirror_zfs/include/sys
History
Tim Chase 25458cbef9 Limit the amount of dnode metadata in the ARC 
Metadata-intensive workloads can cause the ARC to become permanently
filled with dnode_t objects as they're pinned by the VFS layer.
Subsequent data-intensive workloads may only benefit from about
25% of the potential ARC (arc_c_max - arc_meta_limit).

In order to help track metadata usage more precisely, the other_size
metadata arcstat has replaced with dbuf_size, dnode_size and bonus_size.

The new zfs_arc_dnode_limit tunable, which defaults to 10% of
zfs_arc_meta_limit, defines the minimum number of bytes which is desirable
to be consumed by dnodes.  Attempts to evict non-metadata will trigger
async prune tasks if the space used by dnodes exceeds this limit.

The new zfs_arc_dnode_reduce_percent tunable specifies the amount by
which the excess dnode space is attempted to be pruned as a percentage of
the amount by which zfs_arc_dnode_limit is being exceeded.  By default,
it tries to unpin 10% of the dnodes.

The problem of dnode metadata pinning was observed with the following
testing procedure (in this example, zfs_arc_max is set to 4GiB):

    - Create a large number of small files until arc_meta_used exceeds
      arc_meta_limit (3GiB with default tuning) and arc_prune
      starts increasing.

    - Create a 3GiB file with dd.  Observe arc_mata_used.  It will still
      be around 3GiB.

    - Repeatedly read the 3GiB file and observe arc_meta_limit as before.
      It will continue to stay around 3GiB.

With this modification, space for the 3GiB file is gradually made
available as subsequent demands on the ARC are made.  The previous behavior
can be restored by setting zfs_arc_dnode_limit to the same value as the
zfs_arc_meta_limit.

Signed-off-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #4345
Issue #4512
Issue #4773
Closes #4858
2016-07-25 15:26:38 -07:00
..
crypto Illumos Crypto Port module added to enable native encryption in zfs 2016-07-20 10:43:30 -07:00
fm Illumos 3749 - zfs event processing should work on R/O root filesystems 2016-01-12 14:42:32 -08:00
fs OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
arc_impl.h OpenZFS 6513 - partially filled holes lose birth time 2016-06-21 10:55:13 -07:00
arc.h Limit the amount of dnode metadata in the ARC 2016-07-25 15:26:38 -07:00
avl_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
avl.h Illumos 4745 - fix AVL code misspellings 2015-07-10 11:58:37 -07:00
blkptr.h Illumos 4757, 4913 2014-08-01 14:28:05 -07:00
bplist.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
bpobj.h Illumos 5810 - zdb should print details of bpobj 2015-05-11 15:10:24 -07:00
bptree.h Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t 2014-08-06 14:48:41 -07:00
bqueue.h Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
dbuf.h Illumos 6844 - dnode_next_offset can detect fictional holes 2016-04-27 16:24:15 -07:00
ddt.h Add ddt, ddt_entry, and l2arc_hdr caches 2014-01-07 10:33:11 -08:00
dmu_impl.h OpenZFS 6393 - zfs receive a full send as a clone 2016-06-28 13:47:03 -07:00
dmu_objset.h Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
dmu_send.h OpenZFS 2605, 6980, 6902 2016-06-28 13:47:02 -07:00
dmu_traverse.h OpenZFS 2605, 6980, 6902 2016-06-28 13:47:02 -07:00
dmu_tx.h dmu_tx kstat cleanup 2014-03-04 12:22:24 -08:00
dmu_zfetch.h Illumos 5987 - zfs prefetch code needs work 2016-01-12 09:02:33 -08:00
dmu.h OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
dnode.h Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
dsl_bookmark.h Illumos 4368, 4369. 2014-07-29 10:55:29 -07:00
dsl_dataset.h OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
dsl_deadlist.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
dsl_deleg.h Illumos 4368, 4369. 2014-07-29 10:55:29 -07:00
dsl_destroy.h Illumos #3888 2013-11-04 11:18:14 -08:00
dsl_dir.h OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
dsl_pool.h Illumos 5981 - Deadlock in dmu_objset_find_dp 2015-07-06 09:31:35 -07:00
dsl_prop.h Illumos 6171 - dsl_prop_unregister() slows down dataset eviction. 2016-01-12 10:53:12 -08:00
dsl_scan.h Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t 2014-08-06 14:48:41 -07:00
dsl_synctask.h Illumos 4951 - ZFS administrative commands should use reserved space 2015-05-04 09:41:10 -07:00
dsl_userhold.h Illumos #3740 2013-11-04 11:17:48 -08:00
efi_partition.h Ext4's typical GPT partition type not recognized 2015-12-04 09:27:00 -08:00
Makefile.am Illumos Crypto Port module added to enable native encryption in zfs 2016-07-20 10:43:30 -07:00
metaslab_impl.h Remove fastwrite mutex 2016-01-15 15:38:35 -08:00
metaslab.h Illumos 5213 - panic in metaslab_init due to space_map_open returning ENXIO 2014-11-14 15:37:45 -08:00
mntent.h Make zfs mount according to relatime config in dataset 2016-04-05 18:55:59 -07:00
multilist.h Illumos 5497 - lock contention on arcs_mtx 2015-06-11 10:27:25 -07:00
nvpair_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
nvpair.h Replace __va_list with va_list 2014-08-13 10:35:00 -07:00
pathname.h Add pn_alloc()/pn_free() functions 2016-04-21 09:49:25 -07:00
policy.h Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
range_tree.h Illumos #4374 2014-07-30 09:20:35 -07:00
refcount.h Illumos 5045 - use atomic_{inc,dec}_* instead of atomic_add_* 2016-01-15 15:38:36 -08:00
rrwlock.h Illumos 5008 - lock contention (rrw_exit) while running a read only load 2015-07-06 09:34:13 -07:00
sa_impl.h Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
sa.h Prevent SA length overflow 2015-12-30 13:20:12 -08:00
sdt.h Swap DTRACE_PROBE* with Linux tracepoints 2014-11-17 11:13:55 -08:00
spa_boot.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
spa_checksum.h Implementation of AVX2 optimized Fletcher-4 2016-06-02 14:30:51 -07:00
spa_impl.h OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
spa.h Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
space_map.h Illumos 5164-5165 - space map fixes 2014-10-23 15:30:32 -07:00
space_reftree.h Illumos #4101, #4102, #4103, #4105, #4106 2014-07-22 09:39:16 -07:00
trace_acl.h Remove znode's z_uid/z_gid member 2016-07-25 13:21:49 -07:00
trace_arc.h OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
trace_common.h OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
trace_dbgmsg.h SET_ERROR should print strings 2016-01-15 15:38:35 -08:00
trace_dbuf.h SET_ERROR should print strings 2016-01-15 15:38:35 -08:00
trace_dmu.h Fix build failure with Linux 4.1 and FTRACE 2015-07-29 07:35:06 -07:00
trace_dnode.h Fix build failure with Linux 4.1 and FTRACE 2015-07-29 07:35:06 -07:00
trace_multilist.h Fix build failure with Linux 4.1 and FTRACE 2015-08-18 16:47:21 -07:00
trace_txg.h Fix build failure with Linux 4.1 and FTRACE 2015-07-29 07:35:06 -07:00
trace_zil.h Fix build failure with Linux 4.1 and FTRACE 2015-07-29 07:35:06 -07:00
trace_zio.h OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
trace_zrlock.h SET_ERROR should print strings 2016-01-15 15:38:35 -08:00
trace.h Remove duplicate typedefs from trace.h 2015-01-06 16:53:24 -08:00
txg_impl.h Illumos #4045 write throttle & i/o scheduler performance work 2013-12-06 09:32:43 -08:00
txg.h Illumos 4753 - increase number of outstanding async writes when sync task is waiting 2014-09-23 13:50:55 -07:00
u8_textprep_data.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
u8_textprep.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
uberblock_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
uberblock.h Illumos 5347 - idle pool may run itself out of space 2015-07-14 10:35:21 -07:00
uio_impl.h Add basic uio support 2011-02-10 09:21:43 -08:00
unique.h Illumos #3742 2013-11-04 10:55:25 -08:00
uuid.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
vdev_disk.h Fix sync behavior for disk vdevs 2016-07-25 14:24:47 -07:00
vdev_file.h Update all default taskq settings 2015-06-25 08:58:16 -07:00
vdev_impl.h Add -lhHpw options to "zpool iostat" for avg latency, histograms, & queues 2016-05-12 12:36:32 -07:00
vdev_raidz_impl.h RAIDZ parity kstat rework 2016-07-19 16:43:07 -07:00
vdev_raidz.h Fixes and enhancements of SIMD raidz parity 2016-07-19 16:43:07 -07:00
vdev.h Add -lhHpw options to "zpool iostat" for avg latency, histograms, & queues 2016-05-12 12:36:32 -07:00
xvattr.h Add xvattr support 2011-03-02 11:43:50 -08:00
zap_impl.h Illumos 5027 - zfs large block support 2015-05-11 12:23:16 -07:00
zap_leaf.h Illumos 5056 - ZFS deadlock on db_mtx and dn_holds 2015-04-28 16:25:34 -07:00
zap.h OpenZFS 6314 - buffer overflow in dsl_dataset_name 2016-06-28 13:47:03 -07:00
zfeature.h Revert "zhack: Add 'feature disable' command" 2016-05-17 11:52:07 -07:00
zfs_acl.h Illumos #3742 2013-11-04 10:55:25 -08:00
zfs_context.h Illumos Crypto Port module added to enable native encryption in zfs 2016-07-20 10:43:30 -07:00
zfs_ctldir.h Use spa as key besides objsetid for snapentry 2015-12-08 16:38:56 -08:00
zfs_debug.h Add dbgmsg kstat 2015-09-04 16:08:14 -07:00
zfs_delay.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
zfs_dir.h Prototype/structure update for Linux 2011-02-10 09:27:21 -08:00
zfs_fuid.h Prototype/structure update for Linux 2011-02-10 09:27:21 -08:00
zfs_ioctl.h Implement zfs_ioc_recv_new() for OpenZFS 2605 2016-06-28 13:47:03 -07:00
zfs_onexit.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
zfs_rlock.h Use zfs range locks in ztest 2016-05-17 10:40:30 -07:00
zfs_sa.h Illumos 5027 - zfs large block support 2015-05-11 12:23:16 -07:00
zfs_stat.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
zfs_vfsops.h Fix zsb->z_hold_mtx deadlock 2016-01-15 15:33:45 -08:00
zfs_vnops.h Add pn_alloc()/pn_free() functions 2016-04-21 09:49:25 -07:00
zfs_znode.h Remove znode's z_uid/z_gid member 2016-07-25 13:21:49 -07:00
zil_impl.h Illumos 5027 - zfs large block support 2015-05-11 12:23:16 -07:00
zil.h Implement large_dnode pool feature 2016-06-24 13:13:21 -07:00
zio_checksum.h Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
zio_compress.h Illumos #3742 2013-11-04 10:55:25 -08:00
zio_impl.h Illumos #3836 2013-11-05 12:14:56 -08:00
zio_priority.h Add -lhHpw options to "zpool iostat" for avg latency, histograms, & queues 2016-05-12 12:36:32 -07:00
zio.h OpenZFS 6513 - partially filled holes lose birth time 2016-06-21 10:55:13 -07:00
zpl.h Linux 4.7 compat: use iterate_shared for concurrent readdir 2016-05-20 11:09:16 -07:00
zrlock.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
zvol.h OpenZFS 2605, 6980, 6902 2016-06-28 13:47:02 -07:00