Don't hold mutex until release cv in cv_wait

If a thread is holding mutex when doing cv_destroy, it might end up waiting a
thread in cv_wait. The waiter would wake up trying to aquire the same mutex
and cause deadlock.

We solve this by move the mutex_enter to the bottom of cv_wait, so that
the waiter will release the cv first, allowing cv_destroy to succeed and have
a chance to free the mutex.

This would create race condition on the cv_mutex. We use xchg to set and check
it to ensure we won't be harmed by the race. This would result in the cv_mutex
debugging becomes best-effort.

Also, the change reveals a race, which was unlikely before, where we call
mutex_destroy while test threads are still holding the mutex. We use
kthread_stop to make sure the threads are exit before mutex_destroy.

Signed-off-by: Chunwei Chen <tuxoko@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Tim Chase <tim@chase2k.com>
Issue zfsonlinux/zfs#4166
Issue zfsonlinux/zfs#4106
This commit is contained in:
Chunwei Chen
2016-01-06 19:05:24 -08:00
committed by Brian Behlendorf
parent d297a5a3a1
commit e843553d03
2 changed files with 70 additions and 15 deletions
+30
View File
@@ -88,6 +88,9 @@ splat_condvar_test12_thread(void *arg)
ct->ct_thread->comm, atomic_read(&cv->cv_condvar.cv_waiters));
mutex_exit(&cv->cv_mtx);
/* wait for main thread reap us */
while (!kthread_should_stop())
schedule();
return 0;
}
@@ -151,6 +154,12 @@ splat_condvar_test1(struct file *file, void *arg)
/* Wake everything for the failure case */
cv_broadcast(&cv.cv_condvar);
cv_destroy(&cv.cv_condvar);
/* wait for threads to exit */
for (i = 0; i < SPLAT_CONDVAR_TEST_COUNT; i++) {
if (!IS_ERR(ct[i].ct_thread))
kthread_stop(ct[i].ct_thread);
}
mutex_destroy(&cv.cv_mtx);
return rc;
@@ -199,6 +208,12 @@ splat_condvar_test2(struct file *file, void *arg)
/* Wake everything for the failure case */
cv_destroy(&cv.cv_condvar);
/* wait for threads to exit */
for (i = 0; i < SPLAT_CONDVAR_TEST_COUNT; i++) {
if (!IS_ERR(ct[i].ct_thread))
kthread_stop(ct[i].ct_thread);
}
mutex_destroy(&cv.cv_mtx);
return rc;
@@ -234,6 +249,9 @@ splat_condvar_test34_thread(void *arg)
mutex_exit(&cv->cv_mtx);
/* wait for main thread reap us */
while (!kthread_should_stop())
schedule();
return 0;
}
@@ -302,6 +320,12 @@ splat_condvar_test3(struct file *file, void *arg)
/* Wake everything for the failure case */
cv_broadcast(&cv.cv_condvar);
cv_destroy(&cv.cv_condvar);
/* wait for threads to exit */
for (i = 0; i < SPLAT_CONDVAR_TEST_COUNT; i++) {
if (!IS_ERR(ct[i].ct_thread))
kthread_stop(ct[i].ct_thread);
}
mutex_destroy(&cv.cv_mtx);
return rc;
@@ -372,6 +396,12 @@ splat_condvar_test4(struct file *file, void *arg)
/* Wake everything for the failure case */
cv_broadcast(&cv.cv_condvar);
cv_destroy(&cv.cv_condvar);
/* wait for threads to exit */
for (i = 0; i < SPLAT_CONDVAR_TEST_COUNT; i++) {
if (!IS_ERR(ct[i].ct_thread))
kthread_stop(ct[i].ct_thread);
}
mutex_destroy(&cv.cv_mtx);
return rc;