Disable high priority ZIO threads on FreeBSD and Linux

High priority threads are handling ZIL writes.  While there is no
ZIL compression, there is encryption, checksuming and RAIDZ math.
We've found that on large systems 1 taskq with 5 threads can be
a bottleneck for throughput, IOPS or both. Instead of just bumping
number of threads with a risk of overloading CPUs and increasing
latency, switch to using TQ_FRONT mechanism to increase sync write
requests priority within standard write threads.  Do not do it on
Illumos, since its TQ_FRONT implementation is inherently unfair.
FreeBSD and Linux don't have this problem, so we can do it there.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Rob Norris <robn@despairlabs.com>
Signed-off-by: Alexander Motin <mav@FreeBSD.org>
Sponsored-By: iXsystems, Inc.
Closes #16146
This commit is contained in:
Alexander Motin
2024-05-03 12:53:34 -04:00
committed by GitHub
parent 8f1b7a6fa6
commit 04bae5ec95
3 changed files with 16 additions and 9 deletions
+7 -5
View File
@@ -2041,12 +2041,14 @@ zio_taskq_dispatch(zio_t *zio, zio_taskq_type_t q, boolean_t cutinline)
/*
* If this is a high priority I/O, then use the high priority taskq if
* available.
* available or cut the line otherwise.
*/
if ((zio->io_priority == ZIO_PRIORITY_NOW ||
zio->io_priority == ZIO_PRIORITY_SYNC_WRITE) &&
spa->spa_zio_taskq[t][q + 1].stqs_count != 0)
q++;
if (zio->io_priority == ZIO_PRIORITY_SYNC_WRITE) {
if (spa->spa_zio_taskq[t][q + 1].stqs_count != 0)
q++;
else
flags |= TQ_FRONT;
}
ASSERT3U(q, <, ZIO_TASKQ_TYPES);