Illumos 5008 - lock contention (rrw_exit) while running a read only load

5008 lock contention (rrw_exit) while running a read only load
Reviewed by: Matthew Ahrens <matthew.ahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Alex Reece <alex.reece@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Richard Yao <ryao@gentoo.org>
Reviewed by: Saso Kiselkov <skiselkov.ml@gmail.com>
Approved by: Garrett D'Amore <garrett@damore.org>

Porting notes:

This patch ported perfectly cleanly to ZoL.  During testing 100% cached
small-block reads, extreme contention was noticed on rrl->rr_lock from
rrw_exit() due to the frequent entering and leaving ZPL.  Illumos picked
up this patch from FreeBSD and it also helps under Linux.

On a 1-minute 4K cached read test with 10 fio processes pinned to a single
socket on a 4-socket (10 thread per socket) NUMA system, contentions on
rrl->rr_lock were reduced from 508799 to 43085.

Ported-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3555
This commit is contained in:
Alexander Motin
2014-07-18 08:53:38 -08:00
committed by Brian Behlendorf
parent 4bda3bd0e7
commit e16b3fcc61
6 changed files with 126 additions and 13 deletions
+3 -3
View File
@@ -1451,7 +1451,7 @@ zfs_sb_hold(const char *name, void *tag, zfs_sb_t **zsbp, boolean_t writer)
if (get_zfs_sb(name, zsbp) != 0)
error = zfs_sb_create(name, zsbp);
if (error == 0) {
rrw_enter(&(*zsbp)->z_teardown_lock, (writer) ? RW_WRITER :
rrm_enter(&(*zsbp)->z_teardown_lock, (writer) ? RW_WRITER :
RW_READER, tag);
if ((*zsbp)->z_unmounted) {
/*
@@ -1459,7 +1459,7 @@ zfs_sb_hold(const char *name, void *tag, zfs_sb_t **zsbp, boolean_t writer)
* thread should be just about to disassociate the
* objset from the zsb.
*/
rrw_exit(&(*zsbp)->z_teardown_lock, tag);
rrm_exit(&(*zsbp)->z_teardown_lock, tag);
return (SET_ERROR(EBUSY));
}
}
@@ -1469,7 +1469,7 @@ zfs_sb_hold(const char *name, void *tag, zfs_sb_t **zsbp, boolean_t writer)
static void
zfs_sb_rele(zfs_sb_t *zsb, void *tag)
{
rrw_exit(&zsb->z_teardown_lock, tag);
rrm_exit(&zsb->z_teardown_lock, tag);
if (zsb->z_sb) {
deactivate_super(zsb->z_sb);