OpenZFS 9284 - arc_reclaim_thread has 2 jobs

Following the fix for 9018 (Replace kmem_cache_reap_now() with
kmem_cache_reap_soon), the arc_reclaim_thread() no longer blocks
while reaping.  However, the code is still confusing and error-prone,
because this thread has two responsibilities.  We should instead
separate this into two threads each with their own responsibility:

 1. keep `arc_size` under `arc_c`, by calling `arc_adjust()`, which
    improves `arc_is_overflowing()`

 2. keep enough free memory in the system, by calling
    `arc_kmem_reap_now()` plus `arc_shrink()`, which improves
    `arc_available_memory()`.

Furthermore, we can use the zthr infrastructure to separate the
"should we do something" from "do it" parts of the logic, and
normalize the start up / shut down of the threads.

Authored by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Dan McDonald <danmcd@joyent.com>
Reviewed by: Tim Kordas <tim.kordas@joyent.com>
Reviewed by: Tim Chase <tim@chase2k.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by:  Brad Lewis <brad.lewis@delphix.com>
Signed-off-by: Brad Lewis <brad.lewis@delphix.com>

OpenZFS-issue: https://www.illumos.org/issues/9284
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/de753e34f9
Closes #8165
This commit is contained in:
Brad Lewis
2017-03-15 16:41:52 -07:00
committed by Brian Behlendorf
parent 00f198de6b
commit 3ec34e5527
7 changed files with 292 additions and 179 deletions
+27 -1
View File
@@ -47,6 +47,10 @@
* 3] When the zthr is done, it changes the indicator to stopped, allowing
* a new cycle to start.
*
* Besides being awakened by other threads, a zthr can be configured
* during creation to wakeup on its own after a specified interval
* [see zthr_create_timer()].
*
* == ZTHR creation
*
* Every zthr needs three inputs to start running:
@@ -74,6 +78,9 @@
*
* To start a zthr:
* zthr_t *zthr_pointer = zthr_create(checkfunc, func, args);
* or
* zthr_t *zthr_pointer = zthr_create_timer(checkfunc, func,
* args, max_sleep);
*
* After that you should be able to wakeup, cancel, and resume the
* zthr from another thread using zthr_pointer.
@@ -189,7 +196,13 @@ zthr_procedure(void *arg)
mutex_enter(&t->zthr_lock);
} else {
/* go to sleep */
cv_wait_sig(&t->zthr_cv, &t->zthr_lock);
if (t->zthr_wait_time == 0) {
cv_wait_sig(&t->zthr_cv, &t->zthr_lock);
} else {
(void) cv_timedwait_sig_hires(&t->zthr_cv,
&t->zthr_lock, t->zthr_wait_time,
MSEC2NSEC(1), 0);
}
}
}
mutex_exit(&t->zthr_lock);
@@ -199,6 +212,18 @@ zthr_procedure(void *arg)
zthr_t *
zthr_create(zthr_checkfunc_t *checkfunc, zthr_func_t *func, void *arg)
{
return (zthr_create_timer(checkfunc, func, arg, (hrtime_t)0));
}
/*
* Create a zthr with specified maximum sleep time. If the time
* in sleeping state exceeds max_sleep, a wakeup(do the check and
* start working if required) will be triggered.
*/
zthr_t *
zthr_create_timer(zthr_checkfunc_t *checkfunc, zthr_func_t *func,
void *arg, hrtime_t max_sleep)
{
zthr_t *t = kmem_zalloc(sizeof (*t), KM_SLEEP);
mutex_init(&t->zthr_lock, NULL, MUTEX_DEFAULT, NULL);
@@ -208,6 +233,7 @@ zthr_create(zthr_checkfunc_t *checkfunc, zthr_func_t *func, void *arg)
t->zthr_checkfunc = checkfunc;
t->zthr_func = func;
t->zthr_arg = arg;
t->zthr_wait_time = max_sleep;
t->zthr_thread = thread_create(NULL, 0, zthr_procedure, t,
0, &p0, TS_RUN, minclsyspri);