mirror_zfs/module/zfs
Brian Behlendorf 302f753f16 Integrate ARC more tightly with Linux
Under Solaris the ARC was designed to stay one step ahead of the
VM subsystem.  It would attempt to recognize low memory situtions
before they occured and evict data from the cache.  It would also
make assessments about if there was enough free memory to perform
a specific operation.

This was all possible because Solaris exposes a fairly decent
view of the memory state of the system to other kernel threads.
Linux on the other hand does not make this information easily
available.  To avoid extensive modifications to the ARC the SPL
attempts to provide these same interfaces.  While this works it
is not ideal and problems can arise when the ARC and Linux have
different ideas about when your out of memory.  This has manifested
itself in the past as a spinning arc_reclaim_thread.

This patch abandons the emulated Solaris interfaces in favor of
the prefered Linux interface.  That means moving the bulk of the
memory reclaim logic out of the arc_reclaim_thread and in to the
evict driven shrinker callback.  The Linux VM will call this
function when it needs memory.  The ARC is then responsible for
attempting to free the requested amount of memory if possible.

Several interfaces have been modified to accomidate this approach,
however the basic user space implementation remains the same.
The following changes almost exclusively just apply to the kernel
implementation.

* Removed the hdr_recl() reclaim callback which is redundant
  with the broader arc_shrinker_func().

* Reduced arc_grow_retry to 5 seconds from 60.  This is now used
  internally in the ARC with arc_no_grow to indicate that direct
  reclaim was recently performed.  This typically indicates a
  rapid change in memory demands which the kswapd threads were
  unable to keep ahead of.  As long as direct reclaim is happening
  once every 5 seconds arc growth will be paused to avoid further
  contributing to the existing memory pressure.  The more common
  indirect reclaim paths will not set arc_no_grow.

* arc_shrink() has been extended to take the number of bytes by
  which arc_c should be reduced.  This allows for a more granual
  reduction of the arc target.  Since the kernel provides a
  reclaim value to the arc_shrinker_func() this value is used
  instead of 1<<arc_shrink_shift.

* arc_reclaim_needed() has been removed.  It was used to determine
  if the system was under memory pressure and relied extensively
  on Solaris specific VM interfaces.  In most case the new code
  just checks arc_no_grow which indicates that within the last
  arc_grow_retry seconds direct memory reclaim occurred.

* arc_memory_throttle() has been updated to always include the
  amount of evictable memory (arc and page cache) in its free
  space calculations.  This space is largely available in most
  call paths due to direct memory reclaim.

* The Solaris pageout code was also removed to avoid confusion.
  It has always been disabled due to proc_pageout being defined
  as NULL in the Linux port.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2012-04-30 10:03:05 -07:00
..
arc.c Integrate ARC more tightly with Linux 2012-04-30 10:03:05 -07:00
bplist.c Fix gcc missing parenthesis warnings 2010-08-31 08:38:35 -07:00
bpobj.c Update to onnv_147 2010-08-26 14:24:34 -07:00
dbuf.c Illumos #764: panic in zfs:dbuf_sync_list 2011-08-01 12:09:11 -07:00
ddt_zap.c Fix stack ddt_zap_lookup() 2011-05-31 12:17:27 -07:00
ddt.c Fix stack ddt_class_contains() 2011-05-31 12:17:27 -07:00
dmu_diff.c Update to onnv_147 2010-08-26 14:24:34 -07:00
dmu_object.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
dmu_objset.c Export additional dsl symbols 2012-04-11 09:26:55 -07:00
dmu_send.c Illumos #755: dmu_recv_stream builds incomplete guid_to_ds_map 2011-10-18 11:18:14 -07:00
dmu_traverse.c Revert "Fix stack traverse_visitbp()" 2011-05-31 12:17:27 -07:00
dmu_tx.c Illumos #1475: zfs spill block hold can access invalid spill blkptr 2012-04-11 11:46:30 -07:00
dmu_zfetch.c Add missing ZFS tunables 2011-05-04 10:02:37 -07:00
dmu.c Add zfs_mdcomp_disable module option 2012-04-27 16:28:02 -07:00
dnode_sync.c Fix dbuf eviction assertion 2010-08-31 08:38:45 -07:00
dnode.c Improve meta data performance 2011-11-03 10:19:21 -07:00
dsl_dataset.c Add .zfs control directory 2012-03-22 13:03:47 -07:00
dsl_deadlist.c Update core ZFS code from build 121 to build 141. 2010-05-28 13:45:14 -07:00
dsl_deleg.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
dsl_dir.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
dsl_pool.c Add 'dmu_tx' kstats entry 2012-02-27 08:59:10 -08:00
dsl_prop.c Export additional dsl symbols 2012-04-11 09:26:55 -07:00
dsl_scan.c Add missing ZFS tunables 2011-05-04 10:02:37 -07:00
dsl_synctask.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
fm.c Add missing ZFS tunables 2011-05-04 10:02:37 -07:00
gzip.c Fix zmod.h usage in userspace 2010-08-31 08:38:46 -07:00
lzjb.c Fix stack lzjb 2010-08-31 08:38:49 -07:00
Makefile.in Add .zfs control directory 2012-03-22 13:03:47 -07:00
metaslab.c Illumos #1909: disk sync write perf regression when slog is used post oi_148 2012-04-19 16:26:29 -07:00
refcount.c Fix gcc uninitialized variable warnings 2010-08-31 08:38:43 -07:00
rrwlock.c Enable rrwlock.c compilation 2010-12-07 16:05:25 -08:00
sa.c Add sa_spill_rele() interface 2012-03-07 16:28:00 -08:00
sha256.c Add linux sha2 support 2010-08-31 13:41:59 -07:00
spa_boot.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
spa_config.c Prototype/structure update for Linux 2011-02-10 09:27:21 -08:00
spa_errlog.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
spa_history.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
spa_misc.c Cleanup ZFS debug infrastructure 2012-02-02 11:24:30 -08:00
spa.c Illumos #1951: leaking a vdev when removing an l2cache device 2012-04-11 11:32:06 -07:00
space_map.c Update core ZFS code from build 121 to build 141. 2010-05-28 13:45:14 -07:00
txg.c Add 'dmu_tx' kstats entry 2012-02-27 08:59:10 -08:00
uberblock.c Update core ZFS code from build 121 to build 141. 2010-05-28 13:45:14 -07:00
unique.c Fix gcc ident pragma warnings 2010-08-27 15:34:02 -07:00
vdev_cache.c Illumos #175: zfs vdev cache consumes excessive memory 2011-08-01 12:09:11 -07:00
vdev_disk.c Linux 2.6.37 compat, WRITE_FLUSH_FUA 2011-06-17 14:37:26 -07:00
vdev_file.c Illumos #1680: zfs vdev_file_io_start: validate vdev before using vdev_tsd 2012-04-11 11:23:18 -07:00
vdev_label.c Fix gcc uninitialized variable warnings 2010-08-31 08:38:43 -07:00
vdev_mirror.c Fix gcc c90 compliance warnings 2010-08-27 15:28:32 -07:00
vdev_missing.c Update core ZFS code from build 121 to build 141. 2010-05-28 13:45:14 -07:00
vdev_queue.c Add missing ZFS tunables 2011-05-04 10:02:37 -07:00
vdev_raidz.c Fix variables named current 2010-08-31 08:38:44 -07:00
vdev_root.c Fix gcc c90 compliance warnings 2010-08-27 15:28:32 -07:00
vdev.c Illumos #1951: leaking a vdev when removing an l2cache device 2012-04-11 11:32:06 -07:00
zap_leaf.c Fix gcc uninitialized variable warnings 2010-08-31 08:38:43 -07:00
zap_micro.c Export symbols for the full ZAP API 2011-09-27 16:12:36 -07:00
zap.c Fix rw_init() usage 2010-08-31 08:38:46 -07:00
zfs_acl.c Linux compat 2.6.39: mount_nodev() 2011-07-01 13:36:39 -07:00
zfs_byteswap.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
zfs_ctldir.c Account for .zfs ctldir inodes 2012-03-22 15:43:55 -07:00
zfs_debug.c Use spl_debug_* helpers 2012-02-09 16:37:48 -08:00
zfs_dir.c Add .zfs control directory 2012-03-22 13:03:47 -07:00
zfs_fm.c Illumos #1951: leaking a vdev when removing an l2cache device 2012-04-11 11:32:06 -07:00
zfs_fuid.c Drop HAVE_XVATTR macros 2011-03-02 11:44:34 -08:00
zfs_ioctl.c Illumos #1346: zfs incremental receive may leave behind temporary clones 2012-04-11 12:02:27 -07:00
zfs_log.c Drop HAVE_XVATTR macros 2011-03-02 11:44:34 -08:00
zfs_onexit.c Add linux kernel device support 2010-08-31 13:41:50 -07:00
zfs_replay.c Use Linux ATTR_ versions 2011-03-03 11:29:15 -08:00
zfs_rlock.c Range lock performance improvements 2011-03-08 12:44:06 -08:00
zfs_sa.c Use SA_HDL_PRIVATE for SA xattrs 2012-03-02 13:20:48 -08:00
zfs_vfsops.c Fixed a NULL pointer dereference bug in zfs_preumount 2012-04-05 11:29:42 -07:00
zfs_vnops.c Add .zfs control directory 2012-03-22 13:03:47 -07:00
zfs_znode.c Add .zfs control directory 2012-03-22 13:03:47 -07:00
zil.c Illumos #883: ZIL reuse during remount corruption 2011-08-01 12:09:11 -07:00
zio_checksum.c Update core ZFS code from build 121 to build 141. 2010-05-28 13:45:14 -07:00
zio_compress.c Update core ZFS code from build 121 to build 141. 2010-05-28 13:45:14 -07:00
zio_inject.c Add missing ZFS tunables 2011-05-04 10:02:37 -07:00
zio.c Illumos #1909: disk sync write perf regression when slog is used post oi_148 2012-04-19 16:26:29 -07:00
zle.c Update core ZFS code from build 121 to build 141. 2010-05-28 13:45:14 -07:00
zpl_ctldir.c Add .zfs control directory 2012-03-22 13:03:47 -07:00
zpl_export.c Add .zfs control directory 2012-03-22 13:03:47 -07:00
zpl_file.c Support the fallocate() file operation. 2012-02-09 16:19:32 -08:00
zpl_inode.c Add .zfs control directory 2012-03-22 13:03:47 -07:00
zpl_super.c Add .zfs control directory 2012-03-22 13:03:47 -07:00
zpl_xattr.c Add missing NULL in zpl_xattr_handlers 2012-03-15 15:18:29 -07:00
zrlock.c Export ZFS symbols needed by Lustre. 2010-09-17 16:24:15 -07:00
zvol.c Add support for DISCARD to ZVOLs. 2012-02-09 16:19:38 -08:00