mirror of
https://git.proxmox.com/git/mirror_zfs.git
synced 2025-06-28 03:57:35 +03:00
8a2a9db8df
ZFS Performance test concurrency should be lowered for better latency Work by Stephen Blinick. Nightly performance runs typically consist of two levels of concurrency; and both are fairly high. Since the IO runs are to a ZFS filesystem, within a zpool, which is based on some variable number of vdev's, the amount of IO driven to each device is variable. Additionally, different device types (HDD vs SSD, etc) can generally handle a different amount of concurrent IO before saturating. Nevertheless, in practice, it appears that most tests are well past the concurrency saturation point and therefore both perform with the same throughput, the maximum of the device. Because the queuedepth to the device(s) is so high however, the latency is much higher than the best possible at that throughput, and increases linearly with the increase in concurrency. This means that changes in code that impact latency during normal operation (before saturation) may not be apparent when a large component of the measured latency is from the IO sitting in a queue to be serviced. Therefore, changing the concurrency settings is recommended Authored by: Stephen Blinick <stephen.blinick@delphix.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: John Wren Kennedy <jwk404@gmail.com> Ported-by: John Wren Kennedy <jwk404@gmail.com> OpenZFS-issue: https://www.illumos.org/issues/9076 OpenZFS-commit: https://github.com/openzfs/openzfs/pull/562 Upstream bug: DLPX-45477 Closes #7302
100 lines
3.6 KiB
Bash
Executable File
100 lines
3.6 KiB
Bash
Executable File
#!/bin/ksh
|
|
|
|
#
|
|
# This file and its contents are supplied under the terms of the
|
|
# Common Development and Distribution License ("CDDL"), version 1.0.
|
|
# You may only use this file in accordance with the terms of version
|
|
# 1.0 of the CDDL.
|
|
#
|
|
# A full copy of the text of the CDDL should have accompanied this
|
|
# source. A copy of the CDDL is also available via the Internet at
|
|
# http://www.illumos.org/license/CDDL.
|
|
#
|
|
|
|
#
|
|
# Copyright (c) 2015, 2016 by Delphix. All rights reserved.
|
|
#
|
|
|
|
#
|
|
# Description:
|
|
# Trigger fio runs using the sequential_reads job file. The number of runs and
|
|
# data collected is determined by the PERF_* variables. See do_fio_run for
|
|
# details about these variables.
|
|
#
|
|
# The files to read from are created prior to the first fio run, and used
|
|
# for all fio runs. The ARC is cleared with `zinject -a` prior to each run
|
|
# so reads will go to disk.
|
|
#
|
|
# Thread/Concurrency settings:
|
|
# PERF_NTHREADS defines the number of files created in the test filesystem,
|
|
# as well as the number of threads that will simultaneously drive IO to
|
|
# those files. The settings chosen are from measurements in the
|
|
# PerfAutoESX/ZFSPerfESX Environments, selected at concurrency levels that
|
|
# are at peak throughput but lowest latency. Higher concurrency introduces
|
|
# queue time latency and would reduce the impact of code-induced performance
|
|
# regressions.
|
|
#
|
|
|
|
. $STF_SUITE/include/libtest.shlib
|
|
. $STF_SUITE/tests/perf/perf.shlib
|
|
|
|
function cleanup
|
|
{
|
|
# kill fio and iostat
|
|
pkill ${fio##*/}
|
|
pkill ${iostat##*/}
|
|
log_must_busy zfs destroy $TESTFS
|
|
log_must_busy zpool destroy $PERFPOOL
|
|
}
|
|
|
|
trap "log_fail \"Measure IO stats during random read load\"" SIGTERM
|
|
|
|
log_assert "Measure IO stats during sequential read load"
|
|
log_onexit cleanup
|
|
|
|
export TESTFS=$PERFPOOL/testfs
|
|
recreate_perfpool
|
|
log_must zfs create $PERF_FS_OPTS $TESTFS
|
|
|
|
# Aim to fill the pool to 50% capacity while accounting for a 3x compressratio.
|
|
export TOTAL_SIZE=$(($(get_prop avail $TESTFS) * 3 / 2))
|
|
|
|
# Variables for use by fio.
|
|
if [[ -n $PERF_REGRESSION_WEEKLY ]]; then
|
|
export PERF_RUNTIME=${PERF_RUNTIME:-$PERF_RUNTIME_WEEKLY}
|
|
export PERF_RUNTYPE=${PERF_RUNTYPE:-'weekly'}
|
|
export PERF_NTHREADS=${PERF_NTHREADS:-'8 16 32 64'}
|
|
export PERF_SYNC_TYPES=${PERF_SYNC_TYPES:-'1'}
|
|
export PERF_IOSIZES=${PERF_IOSIZES:-'64k 128k 1m'}
|
|
elif [[ -n $PERF_REGRESSION_NIGHTLY ]]; then
|
|
export PERF_RUNTIME=${PERF_RUNTIME:-$PERF_RUNTIME_NIGHTLY}
|
|
export PERF_RUNTYPE=${PERF_RUNTYPE:-'nightly'}
|
|
export PERF_NTHREADS=${PERF_NTHREADS:-'8 16'}
|
|
export PERF_SYNC_TYPES=${PERF_SYNC_TYPES:-'1'}
|
|
export PERF_IOSIZES=${PERF_IOSIZES:-'128k 1m'}
|
|
fi
|
|
|
|
# Layout the files to be used by the read tests. Create as many files as the
|
|
# largest number of threads. An fio run with fewer threads will use a subset
|
|
# of the available files.
|
|
export NUMJOBS=$(get_max $PERF_NTHREADS)
|
|
export FILE_SIZE=$((TOTAL_SIZE / NUMJOBS))
|
|
log_must fio $FIO_SCRIPTS/mkfiles.fio
|
|
|
|
# Set up the scripts and output files that will log performance data.
|
|
lun_list=$(pool_to_lun_list $PERFPOOL)
|
|
log_note "Collecting backend IO stats with lun list $lun_list"
|
|
if is_linux; then
|
|
export collect_scripts=("zpool iostat -lpvyL $PERFPOOL 1" "zpool.iostat"
|
|
"$PERF_SCRIPTS/prefetch_io.sh $PERFPOOL 1" "prefetch" "vmstat 1"
|
|
"vmstat" "mpstat -P ALL 1" "mpstat" "iostat -dxyz 1" "iostat")
|
|
else
|
|
export collect_scripts=("$PERF_SCRIPTS/io.d $PERFPOOL $lun_list 1" "io"
|
|
"$PERF_SCRIPTS/prefetch_io.d $PERFPOOL 1" "prefetch" "vmstat 1" "vmstat"
|
|
"mpstat 1" "mpstat" "iostat -xcnz 1" "iostat")
|
|
fi
|
|
|
|
log_note "Sequential reads with $PERF_RUNTYPE settings"
|
|
do_fio_run sequential_reads.fio false true
|
|
log_pass "Measure IO stats during sequential read load"
|