mirror of
https://git.proxmox.com/git/mirror_zfs.git
synced 2024-11-18 02:20:59 +03:00
6bb24f4dc7
Add the ZFS Test Suite and test-runner framework from illumos.
This is a continuation of the work done by Turbo Fredriksson to
port the ZFS Test Suite to Linux. While this work was originally
conceived as a stand alone project integrating it directly with
the ZoL source tree has several advantages:
* Allows the ZFS Test Suite to be packaged in zfs-test package.
* Facilitates easy integration with the CI testing.
* Users can locally run the ZFS Test Suite to validate ZFS.
This testing should ONLY be done on a dedicated test system
because the ZFS Test Suite in its current form is destructive.
* Allows the ZFS Test Suite to be run directly in the ZoL source
tree enabled developers to iterate quickly during development.
* Developers can easily add/modify tests in the framework as
features are added or functionality is changed. The tests
will then always be in sync with the implementation.
Full documentation for how to run the ZFS Test Suite is available
in the tests/README.md file.
Warning: This test suite is designed to be run on a dedicated test
system. It will make modifications to the system including, but
not limited to, the following.
* Adding new users
* Adding new groups
* Modifying the following /proc files:
* /proc/sys/kernel/core_pattern
* /proc/sys/kernel/core_uses_pid
* Creating directories under /
Notes:
* Not all of the test cases are expected to pass and by default
these test cases are disabled. The failures are primarily due
to assumption made for illumos which are invalid under Linux.
* When updating these test cases it should be done in as generic
a way as possible so the patch can be submitted back upstream.
Most existing library functions have been updated to be Linux
aware, and the following functions and variables have been added.
* Functions:
* is_linux - Used to wrap a Linux specific section.
* block_device_wait - Waits for block devices to be added to /dev/.
* Variables: Linux Illumos
* ZVOL_DEVDIR "/dev/zvol" "/dev/zvol/dsk"
* ZVOL_RDEVDIR "/dev/zvol" "/dev/zvol/rdsk"
* DEV_DSKDIR "/dev" "/dev/dsk"
* DEV_RDSKDIR "/dev" "/dev/rdsk"
* NEWFS_DEFAULT_FS "ext2" "ufs"
* Many of the disabled test cases fail because 'zfs/zpool destroy'
returns EBUSY. This is largely causes by the asynchronous nature
of device handling on Linux and is expected, the impacted test
cases will need to be updated to handle this.
* There are several test cases which have been disabled because
they can trigger a deadlock. A primary example of this is to
recursively create zpools within zpools. These tests have been
disabled until the root issue can be addressed.
* Illumos specific utilities such as (mkfile) should be added to
the tests/zfs-tests/cmd/ directory. Custom programs required by
the test scripts can also be added here.
* SELinux should be either is permissive mode or disabled when
running the tests. The test cases should be updated to conform
to a standard policy.
* Redundant test functionality has been removed (zfault.sh).
* Existing test scripts (zconfig.sh) should be migrated to use
the framework for consistency and ease of testing.
* The DISKS environment variable currently only supports loopback
devices because of how the ZFS Test Suite expects partitions to
be named (p1, p2, etc). Support must be added to generate the
correct partition name based on the device location and name.
* The ZFS Test Suite is part of the illumos code base at:
https://github.com/illumos/illumos-gate/tree/master/usr/src/test
Original-patch-by: Turbo Fredriksson <turbo@bayour.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Closes #6
Closes #1534
371 lines
8.1 KiB
Groff
371 lines
8.1 KiB
Groff
.\"
|
|
.\" This file and its contents are supplied under the terms of the
|
|
.\" Common Development and Distribution License ("CDDL"), version 1.0.
|
|
.\" You may only use this file in accordance with the terms of version
|
|
.\" 1.0 of the CDDL.
|
|
.\"
|
|
.\" A full copy of the text of the CDDL should have accompanied this
|
|
.\" source. A copy of the CDDL is also available via the Internet at
|
|
.\" http://www.illumos.org/license/CDDL.
|
|
.\"
|
|
.\"
|
|
.\" Copyright (c) 2012 by Delphix. All rights reserved.
|
|
.\"
|
|
.TH run 1 "23 Sep 2012"
|
|
.SH NAME
|
|
run \- find, execute, and log the results of tests
|
|
.SH SYNOPSIS
|
|
.LP
|
|
.nf
|
|
\fBrun\fR [\fB-dgq] [\fB-o\fR \fIoutputdir\fR] [\fB-pP\fR \fIscript\fR] [\fB-t\fR \fIseconds\fR] [\fB-uxX\fR \fIusername\fR]
|
|
\fIpathname\fR ...
|
|
.fi
|
|
|
|
.LP
|
|
.nf
|
|
\fBrun\fR \fB-w\fR \fIrunfile\fR [\fB-gq\fR] [\fB-o\fR \fIoutputdir\fR] [\fB-pP\fR \fIscript\fR] [\fB-t\fR \fIseconds\fR]
|
|
[\fB-uxX\fR \fIusername\fR] \fIpathname\fR ...
|
|
.fi
|
|
|
|
.LP
|
|
.nf
|
|
\fBrun\fR \fB-c\fR \fIrunfile\fR [\fB-dq\fR]
|
|
.fi
|
|
|
|
.LP
|
|
.nf
|
|
\fBrun\fR [\fB-h\fR]
|
|
.fi
|
|
|
|
.SH DESCRIPTION
|
|
.sp
|
|
.LP
|
|
The \fBrun\fR command has three basic modes of operation. With neither the
|
|
\fB-c\fR nor the \fB-w\fR option, \fBrun\fR processes the arguments provided on
|
|
the command line, adding them to the list for this run. If a specified
|
|
\fIpathname\fR is an executable file, it is added as a test. If a specified
|
|
\fIpathname\fR is a directory, the behavior depends upon the \fB-g\fR option.
|
|
If \fB-g\fR is specified, the directory is treated as a test group. See the
|
|
section on "Test Groups" below. Without the \fB-g\fR option, \fBrun\fR simply
|
|
descends into the directory looking for executable files. The tests are then
|
|
executed, and the results are logged.
|
|
|
|
With the \fB-w\fR option, \fBrun\fR finds tests in the manner described above.
|
|
Rather than executing the tests and logging the results, the test configuration
|
|
is stored in a \fIrunfile\fR which can be used in future invocations, or edited
|
|
to modify which tests are executed and which options are applied. Options
|
|
included on the command line with \fB-w\fR become defaults in the
|
|
\fIrunfile\fR.
|
|
|
|
With the \fB-c\fR option, \fBrun\fR parses a \fIrunfile\fR, which can specify a
|
|
series of tests and test groups to be executed. The tests are then executed,
|
|
and the results are logged.
|
|
.sp
|
|
.SS "Test Groups"
|
|
.sp
|
|
.LP
|
|
A test group is comprised of a set of executable files, all of which exist in
|
|
one directory. The options specified on the command line or in a \fIrunfile\fR
|
|
apply to individual tests in the group. The exception is options pertaining to
|
|
pre and post scripts, which act on all tests as a group. Rather than running
|
|
before and after each test, these scripts are run only once each at the start
|
|
and end of the test group.
|
|
.SS "Test Execution"
|
|
.sp
|
|
.LP
|
|
The specified tests run serially, and are typically assigned results according
|
|
to exit values. Tests that exit zero and non-zero are marked "PASS" and "FAIL"
|
|
respectively. When a pre script fails for a test group, only the post script is
|
|
executed, and the remaining tests are marked "SKIPPED." Any test that exceeds
|
|
its \fItimeout\fR is terminated, and marked "KILLED."
|
|
|
|
By default, tests are executed with the credentials of the \fBrun\fR script.
|
|
Executing tests with other credentials is done via \fBsudo\fR(1m), which must
|
|
be configured to allow execution without prompting for a password. Environment
|
|
variables from the calling shell are available to individual tests. During test
|
|
execution, the working directory is changed to \fIoutputdir\fR.
|
|
.SS "Output Logging"
|
|
.sp
|
|
.LP
|
|
By default, \fBrun\fR will print one line on standard output at the conclusion
|
|
of each test indicating the test name, result and elapsed time. Additionally,
|
|
for each invocation of \fBrun\fR, a directory is created using the ISO 8601
|
|
date format. Within this directory is a file named \fIlog\fR containing all the
|
|
test output with timestamps, and a directory for each test. Within the test
|
|
directories, there is one file each for standard output, standard error and
|
|
merged output. The default location for the \fIoutputdir\fR is
|
|
\fI/var/tmp/test_results\fR.
|
|
.SS "Runfiles"
|
|
.sp
|
|
.LP
|
|
The \fIrunfile\fR is an ini style configuration file that describes a test run.
|
|
The file has one section named "DEFAULT," which contains configuration option
|
|
names and their values in "name = value" format. The values in this section
|
|
apply to all the subsequent sections, unless they are also specified there, in
|
|
which case the default is overridden. The remaining section names are the
|
|
absolute pathnames of files and direcotries, describing tests and test groups
|
|
respectively. The legal option names are:
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fBoutputdir\fR = \fIpathname\fR
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
The name of the directory that holds test logs.
|
|
.RE
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fBpre\fR = \fIscript\fR
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
Run \fIscript\fR prior to the test or test group.
|
|
.RE
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fBpre_user\fR = \fIusername\fR
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
Execute the pre script as \fIusername\fR.
|
|
.RE
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fBpost\fR = \fIscript\fR
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
Run \fIscript\fR after the test or test group.
|
|
.RE
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fBpost_user\fR = \fIusername\fR
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
Execute the post script as \fIusername\fR.
|
|
.RE
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fBquiet\fR = [\fITrue\fR|\fIFalse\fR]
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
If set to True, only the results summary is printed to standard out.
|
|
.RE
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fBtests\fR = [\fI'filename'\fR [,...]]
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
Specify a list of \fIfilenames\fR for this test group. Only the basename of the
|
|
absolute path is required. This option is only valid for test groups, and each
|
|
\fIfilename\fR must be single quoted.
|
|
.RE
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fBtimeout\fR = \fIn\fR
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
A timeout value of \fIn\fR seconds.
|
|
.RE
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fBuser\fR = \fIusername\fR
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
Execute the test or test group as \fIusername\fR.
|
|
.RE
|
|
|
|
.SH OPTIONS
|
|
.sp
|
|
.LP
|
|
The following options are available for the \fBrun\fR command.
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fB-c\fR \fIrunfile\fR
|
|
.ad
|
|
.RS 6n
|
|
Specify a \fIrunfile\fR to be consumed by the run command.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-d\fR
|
|
.ad
|
|
.RS 6n
|
|
Dry run mode. Execute no tests, but print a description of each test that would
|
|
have been run.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-g\fR
|
|
.ad
|
|
.RS 6n
|
|
Create test groups from any directories found while searching for tests.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-o\fR \fIoutputdir\fR
|
|
.ad
|
|
.RS 6n
|
|
Specify the directory in which to write test results.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-p\fR \fIscript\fR
|
|
.ad
|
|
.RS 6n
|
|
Run \fIscript\fR prior to any test or test group.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-P\fR \fIscript\fR
|
|
.ad
|
|
.RS 6n
|
|
Run \fIscript\fR after any test or test group.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-q\fR
|
|
.ad
|
|
.RS 6n
|
|
Print only the results sumary to the standard output.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-t\fR \fIn\fR
|
|
.ad
|
|
.RS 6n
|
|
Specify a timeout value of \fIn\fR seconds per test.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-u\fR \fIusername\fR
|
|
.ad
|
|
.RS 6n
|
|
Execute tests or test groups as \fIusername\fR.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-w\fR \fIrunfile\fR
|
|
.ad
|
|
.RS 6n
|
|
Specify the name of the \fIrunfile\fR to create.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-x\fR \fIusername\fR
|
|
.ad
|
|
.RS 6n
|
|
Execute the pre script as \fIusername\fR.
|
|
.RE
|
|
|
|
.ne 2
|
|
.na
|
|
\fB-X\fR \fIusername\fR
|
|
.ad
|
|
.RS 6n
|
|
Execute the post script as \fIusername\fR.
|
|
.RE
|
|
|
|
.SH EXAMPLES
|
|
.LP
|
|
\fBExample 1\fR Running ad-hoc tests.
|
|
.sp
|
|
.LP
|
|
This example demonstrates the simplest invocation of \fBrun\fR.
|
|
|
|
.sp
|
|
.in +2
|
|
.nf
|
|
% \fBrun my-tests\fR
|
|
Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS]
|
|
Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS]
|
|
Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS]
|
|
|
|
Results Summary
|
|
PASS 3
|
|
|
|
Running Time: 00:00:07
|
|
Percent passed: 100.0%
|
|
Log directory: /var/tmp/test_results/20120923T180654
|
|
.fi
|
|
.in -2
|
|
|
|
.LP
|
|
\fBExample 2\fR Creating a \fIrunfile\fR for future use.
|
|
.sp
|
|
.LP
|
|
This example demonstrates creating a \fIrunfile\fR with non default options.
|
|
|
|
.sp
|
|
.in +2
|
|
.nf
|
|
% \fBrun -p setup -x root -g -w new-tests.run new-tests\fR
|
|
% \fBcat new-tests.run\fR
|
|
[DEFAULT]
|
|
pre = setup
|
|
post_user =
|
|
quiet = False
|
|
user =
|
|
timeout = 60
|
|
post =
|
|
pre_user = root
|
|
outputdir = /var/tmp/test_results
|
|
|
|
[/home/jkennedy/new-tests]
|
|
tests = ['test-01', 'test-02', 'test-03']
|
|
.fi
|
|
.in -2
|
|
|
|
.SH EXIT STATUS
|
|
.sp
|
|
.LP
|
|
The following exit values are returned:
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fB\fB0\fR\fR
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
Successful completion.
|
|
.RE
|
|
.sp
|
|
.ne 2
|
|
.na
|
|
\fB\fB1\fR\fR
|
|
.ad
|
|
.sp .6
|
|
.RS 4n
|
|
An error occurred.
|
|
.RE
|
|
|
|
.SH SEE ALSO
|
|
.sp
|
|
.LP
|
|
\fBsudo\fR(1m)
|