mirror_zfs/include
Paul Dagnelie f09fda5071 Cap metaslab memory usage
On systems with large amounts of storage and high fragmentation, a huge 
amount of space can be used by storing metaslab range trees. Since 
metaslabs are only unloaded during a txg sync, and only if they have 
been inactive for 8 txgs, it is possible to get into a state where all 
of the system's memory is consumed by range trees and metaslabs, and 
txgs cannot sync. While ZFS knows how to evict ARC data when needed, 
it has no such mechanism for range tree data. This can result in boot 
hangs for some system configurations.

First, we add the ability to unload metaslabs outside of syncing 
context. Second, we store a multilist of all loaded metaslabs, sorted 
by their selection txg, so we can quickly identify the oldest 
metaslabs.  We use a multilist to reduce lock contention during heavy 
write workloads. Finally, we add logic that will unload a metaslab 
when we're loading a new metaslab, if we're using more than a certain 
fraction of the available memory on range trees.

Reviewed-by: Matt Ahrens <mahrens@delphix.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed-by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #9128
2019-08-16 09:08:21 -06:00
..
linux Fix CONFIG_X86_DEBUG_FPU build failure 2019-07-17 09:14:36 -07:00
spl Drop KMC_NOEMERGENCY 2019-08-13 07:46:12 -06:00
sys Cap metaslab memory usage 2019-08-16 09:08:21 -06:00
.gitignore Create /proc/sys/kernel/spl/gitrev with git hash 2018-10-08 21:57:02 -07:00
libnvpair.h Add JSON output support to channel programs 2018-03-19 12:40:58 -07:00
libuutil_common.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
libuutil_impl.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
libuutil.h Correct cppcheck errors 2017-09-19 12:17:29 -07:00
libzfs_core.h Remove code for zfs remap 2019-06-24 16:44:01 -07:00
libzfs_impl.h OpenZFS 8115 - parallel zfs mount 2018-11-15 11:33:58 -08:00
libzfs.h Race condition between spa async threads and export 2019-07-18 13:02:33 -07:00
libzutil.h Fix libudev dependency in libzutil 2018-11-06 17:47:52 -08:00
Makefile.am Add libzutil for libzfs or libzpool consumers 2018-11-05 11:22:33 -08:00
thread_pool.h Add libtpool (thread pools) 2017-08-09 15:31:08 -07:00
zfeature_common.h Fast Clone Deletion 2019-07-26 10:54:14 -07:00
zfs_comutil.h OpenZFS 9337 - zfs get all is slow due to uncached metadata 2018-07-12 10:49:27 -07:00
zfs_deleg.h Remove code for zfs remap 2019-06-24 16:44:01 -07:00
zfs_fletcher.h DLPX-44812 integrate EP-220 large memory scalability 2016-11-29 14:34:27 -08:00
zfs_namecheck.h Restrict filesystem creation if name referred either '.' or '..' 2019-06-13 08:56:15 -07:00
zfs_prop.h Add zfs module feature and property info to sysfs 2018-09-02 12:09:53 -07:00