c/mirror_zfs
1
0
Fork 0
mirror of https://git.proxmox.com/git/mirror_zfs.git synced 2024-11-18 02:20:59 +03:00
Code Issues Packages Projects Releases Wiki Activity
380b08098e
mirror_zfs/tests/zfs-tests/include/commands.cfg

229 lines
2.9 KiB
INI
Raw Normal View History

OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
#
Tests for btree implementation used by range trees Additional test cases for the btree implementation, see #9181. Reviewed-by: Paul Dagnelie <pcd@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: John Kennedy <john.kennedy@delphix.com> Closes #9717
2019-12-19 22:53:55 +03:00
# Copyright (c) 2016, 2019 by Delphix. All rights reserved.
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
# These variables are used by zfs-tests.sh to constrain which utilities
# may be used by the suite. The suite will create a directory which is
# the only element of $PATH and create symlinks from that dir to the
# binaries listed below.
#
# Please keep the contents of each variable sorted for ease of reading
# and maintenance.
#
tests: clean out unused/single-use/useless commands from the list Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Closes #13259
2022-03-14 03:41:03 +03:00
export SYSTEM_FILES_COMMON='awk
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
basename
bc
bunzip2
bzcat
cat
chgrp
chmod
chown
ZTS: Move cksum to common system commands The cksum command is used by delegate tests. We have it on FreeBSD, so it should not have been moved to the Linux commands list. Move it back to the common commands list. Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Closes #10007
2020-02-16 23:49:49 +03:00
cksum
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
cmp
cp
cpio
cut
date
dd
df
diff
dirname
zfstest: add dmesg command to $PATH Error example in `zfs_list_007_pos`: `sudo: dmesg: command not found` Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: George Melikov <mail@gmelikov.ru> Closes #6024
2017-04-18 19:44:17 +03:00
dmesg
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
du
echo
ZTS: Use ksh and current environment for user_run The current user_run often does not work as expected. Commands are run in a different shell, with a different environment, and all output is discarded. Simplify user_run to retain the current environment, eliminate eval, and feed the command string into ksh. Enhance the logging for user_run so we can see out and err. Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Closes #11185
2021-03-11 23:01:58 +03:00
env
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
expr
false
file
find
fio
getconf
getent
getfacl
grep
gunzip
gzip
head
hostname
id
iostat
kill
ksh
Add `--enable-asan` and `--enable-ubsan` switches `configure` now accepts `--enable-asan` and `--enable-ubsan` switches which results in passing `-fsanitize=address` and `-fsanitize=undefined`, respectively, to the compiler. Those flags are enabled in GitHub workflows for ZTS and zloop. Errors reported by both instrumentations are corrected, except for: - Memory leak reporting is (temporarily) suppressed. The cost of fixing them is relatively high compared to the gains. - Checksum computing functions in `module/zcommon/zfs_fletcher*` have UBSan errors suppressed. It is completely impractical to enforce 64-byte payload alignment there due to performance impact. - There's no ASan heap poisoning in `module/zstd/lib/zstd.c`. A custom memory allocator is used there rendering that measure unfeasible. - Memory leaks detection has to be suppressed for `cmd/zvol_id`. `zvol_id` is run by udev with the help of `ptrace(2)`. Tracing is incompatible with memory leaks detection. Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: szubersk <szuberskidamian@gmail.com> Closes #12928
2022-02-04 01:35:38 +03:00
ldd
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
ln
ls
mkdir
mknod
tests: clean out unused/single-use/useless commands from the list Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Closes #13259
2022-03-14 03:41:03 +03:00
mkfifo
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
mktemp
mount
mv
net
OpenZFS 9102 - zfs should be able to initialize storage devices PROBLEM ======== The first access to a block incurs a performance penalty on some platforms (e.g. AWS's EBS, VMware VMDKs). Therefore we recommend that volumes are "thick provisioned", where supported by the platform (VMware). This can create a large delay in getting a new virtual machines up and running (or adding storage to an existing Engine). If the thick provision step is omitted, write performance will be suboptimal until all blocks on the LUN have been written. SOLUTION ========= This feature introduces a way to 'initialize' the disks at install or in the background to make sure we don't incur this first read penalty. When an entire LUN is added to ZFS, we make all space available immediately, and allow ZFS to find unallocated space and zero it out. This works with concurrent writes to arbitrary offsets, ensuring that we don't zero out something that has been (or is in the middle of being) written. This scheme can also be applied to existing pools (affecting only free regions on the vdev). Detailed design: - new subcommand:zpool initialize [-cs] <pool> [<vdev> ...] - start, suspend, or cancel initialization - Creates new open-context thread for each vdev - Thread iterates through all metaslabs in this vdev - Each metaslab: - select a metaslab - load the metaslab - mark the metaslab as being zeroed - walk all free ranges within that metaslab and translate them to ranges on the leaf vdev - issue a "zeroing" I/O on the leaf vdev that corresponds to a free range on the metaslab we're working on - continue until all free ranges for this metaslab have been "zeroed" - reset/unmark the metaslab being zeroed - if more metaslabs exist, then repeat above tasks. - if no more metaslabs, then we're done. - progress for the initialization is stored on-disk in the vdev’s leaf zap object. The following information is stored: - the last offset that has been initialized - the state of the initialization process (i.e. active, suspended, or canceled) - the start time for the initialization - progress is reported via the zpool status command and shows information for each of the vdevs that are initializing Porting notes: - Added zfs_initialize_value module parameter to set the pattern written by "zpool initialize". - Added zfs_vdev_{initializing,removal}_{min,max}_active module options. Authored by: George Wilson <george.wilson@delphix.com> Reviewed by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Reviewed by: loli10K <ezomori.nozomu@gmail.com> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Approved by: Richard Lowe <richlowe@richlowe.net> Signed-off-by: Tim Chase <tim@chase2k.com> Ported-by: Tim Chase <tim@chase2k.com> OpenZFS-issue: https://www.illumos.org/issues/9102 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/c3963210eb Closes #8230
2018-12-19 17:54:59 +03:00
od
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
openssl
pam: implement a zfs_key pam module Implements a pam module for automatically loading zfs encryption keys for home datasets. The pam module: - loads a zfs key and mounts the dataset when a session opens. - unmounts the dataset and unloads the key when the session closes. - when the user is logged on and changes the password, the module changes the encryption key. Reviewed-by: Richard Laager <rlaager@wiktel.com> Reviewed-by: @jengelh <jengelh@inai.de> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Felix Dörre <felix@dogcraft.de> Closes #9886 Closes #9903
2020-06-25 04:45:44 +03:00
pamtester
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
pax
pgrep
ping
pkill
printf
ps
Add Python 3 rewrite of arc_summary.py Add new script arc_summary3.py as a complete rewrite of the arc_summary.py tool (see issue #6873) Add new options: -g/--graph - Display crude graphic representation of ARC status and quit -r/--raw - Print all available information as minimally formatted list (for grep) -s/--section - Print a single section. This replaces -p/--page, which is kept for backwards use but marked as depreciated Add new sections with information on ZIL and SPL. Notify user if sections L2ARC and VDEV are skipped instead of failing silently. Add warning that -p/--page option is depreciated. Developed for Python 3.5. Reviewed-by: Richard Laager <rlaager@wiktel.com> Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com> Reviewed by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Scot W. Stevenson <scot.stevenson@gmail.com> Closes #6873 Closes #6892
2018-02-28 19:52:34 +03:00
python3
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
readlink
rm
rmdir
ZTS: switch to rsync for directory diffs While "diff -r" is the most straightforward way of comparing directory trees for differences, it has two major issues: * File metadata is not compared, which means that subtle bugs may be missed even if a test is written that exercises the buggy behaviour. * diff(1) doesn't know how to compare special files -- it assumes they are always different, which means that a test using diff(1) on special files will always fail (resulting in such tests not being added). rsync can be used in a very similar manner to diff (with the -ni flags), but has the additional benefit of being able to detect and resolve many more differences between directory trees. In addition, rsync has a standard set of features and flags while diffs feature set depends on whether you're using GNU or BSD binutils. Note that for several of the test cases we expect that file timestamps will not match. For example, the ctime for a file creation or modify event is stored in the intent log but not the mtime. Thus when replaying the log the correct ctime is set but the current mtime is used. This is the expected behavior, so to prevent these tests from failing, there's a replay_directory_diff function which ignores those kinds of changes. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Aleksa Sarai <cyphar@cyphar.com> Closes #12588
2022-03-01 21:05:32 +03:00
rsync
OpenZFS 9185 - Enable testing over NFS in ZFS performance tests This change makes additions to the ZFS test suite that allows the performance tests to run over NFS. The test is run and performance data collected from the server side, while IO is generated on the NFS client. This has been tested with Linux and illumos NFS clients. Authored by: Ahmed Ghanem <ahmedg@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: Kevin Greene <kevin.greene@delphix.com> Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: John Kennedy <john.kennedy@delphix.com> OpenZFS-issue: https://www.illumos.org/issues/9185 Closes #8367
2018-02-12 02:11:59 +03:00
scp
Colorize zpool status output If the ZFS_COLOR env variable is set, then use ANSI color output in zpool status: - Column headers are bold - Degraded or offline pools/vdevs are yellow - Non-zero error counters and faulted vdevs/pools are red - The 'status:' and 'action:' sections are yellow if they're displaying a warning. This also includes a new 'faketty' function in libtest.shlib that is compatible with FreeBSD (code provided by @freqlabs). Reviewed-by: Jorgen Lundman <lundman@lundman.net> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@ixsystems.com> Signed-off-by: Tony Hutter <hutter2@llnl.gov> Closes #9340
2019-12-20 03:26:07 +03:00
script
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
sed
seq
setfacl
sh
sleep
sort
OpenZFS 9185 - Enable testing over NFS in ZFS performance tests This change makes additions to the ZFS test suite that allows the performance tests to run over NFS. The test is run and performance data collected from the server side, while IO is generated on the NFS client. This has been tested with Linux and illumos NFS clients. Authored by: Ahmed Ghanem <ahmedg@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: Kevin Greene <kevin.greene@delphix.com> Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: John Kennedy <john.kennedy@delphix.com> OpenZFS-issue: https://www.illumos.org/issues/9185 Closes #8367
2018-02-12 02:11:59 +03:00
ssh
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
stat
strings
sudo
Enable additional test cases Enable additional test cases, in most cases this required a few minor modifications to the test scripts. In a few cases a real bug was uncovered and fixed. And in a handful of cases where pools are layered on pools the test case will be skipped until this is supported. Details below for each test case. * zpool_add_004_pos - Skip test on Linux until adding zvols to pools is fully supported and deadlock free. * zpool_add_005_pos.ksh - Skip dumpadm portion of the test which isn't relevant for Linux. The find_vfstab_dev, find_mnttab_dev, and save_dump_dev functions were updated accordingly for Linux. Add O_EXCL to the in-use check to prevent the -f (force) option from working for mounted filesystems and improve the resulting error. * zpool_add_006_pos - Update test case such that it doesn't depend on nested pools. Switch to truncate from mkfile to reduce space requirements and speed up the test case. * zpool_clear_001_pos - Speed up test case by filling filesystem to 25% capacity. * zpool_create_002_pos, zpool_create_004_pos - Use sparse files for file vdevs in order to avoid increasing the partition size. * zpool_create_006_pos - 6ba1ce9 allows raidz+mirror configs with similar redundancy. Updating the valid_args and forced_args cases. * zpool_create_008_pos - Disable overlapping partition portion. * zpool_create_011_neg - Fix to correctly create the extra partition. Modified zpool_vdev.c to use fstat64_blk() wrapper which includes the st_size even for block devices. * zpool_create_012_neg - Updated to properly find swap devices. * zpool_create_014_neg, zpool_create_015_neg - Updated to use swap_setup() and swap_cleanup() wrappers which do the right thing on Linux and Illumos. Removed '-n' option which succeeds under Linux due to differences in the in-use checks. * zpool_create_016_pos.ksh - Skipped test case isn't useful. * zpool_create_020_pos - Added missing / to cleanup() function. Remove cache file prior to test to ensure a clean environment and avoid false positives. * zpool_destroy_001_pos - Removed test case which creates a pool on a zvol. This is more likely to deadlock under Linux and has never been completely supported on any platform. * zpool_destroy_002_pos - 'zpool destroy -f' is unsupported on Linux. Mount point must not be busy in order to unmount them. * zfs_destroy_001_pos - Handle EBUSY error which can occur with volumes when racing with udev. * zpool_expand_001_pos, zpool_expand_003_neg - Skip test on Linux until adding zvols to pools is fully supported and deadlock free. The test could be modified to use loop-back devices but it would be preferable to use the test case as is for improved coverage. * zpool_export_004_pos - Updated test case to such that it doesn't depend on nested pools. Normal file vdev under /var/tmp are fine. * zpool_import_all_001_pos - Updated to skip partition 1, which is known as slice 2, on Illumos. This prevents overwriting the default TESTPOOL which was causing the failure. * zpool_import_002_pos, zpool_import_012_pos - No changes needed. * zpool_remove_003_pos - No changes needed * zpool_upgrade_002_pos, zpool_upgrade_004_pos - Root cause addressed by upstream OpenZFS commit 3b7f360. * zpool_upgrade_007_pos - Disabled in test case due to known failure. Opened issue https://github.com/zfsonlinux/zfs/issues/6112 * zvol_misc_002_pos - Updated to to use ext2. * zvol_misc_001_neg, zvol_misc_003_neg, zvol_misc_004_pos, zvol_misc_005_neg, zvol_misc_006_pos - Moved to skip list, these test case could be updated to use Linux's crash dump facility. * zvol_swap_* - Updated to use swap_setup/swap_cleanup helpers. File creation switched from /tmp to /var/tmp. Enabled minimal useful tests for Linux, skip test cases which aren't applicable. Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov> Reviewed-by: loli10K <ezomori.nozomu@gmail.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #3484 Issue #5634 Issue #2437 Issue #5202 Issue #4034 Closes #6095
2017-05-12 00:27:57 +03:00
swapoff
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
swapon
sync
tail
tar
timeout
touch
tr
true
truncate
umount
uname
Avoid posting duplicate zpool events Duplicate io and checksum ereport events can misrepresent that things are worse than they seem. Ideally the zpool events and the corresponding vdev stat error counts in a zpool status should be for unique errors -- not the same error being counted over and over. This can be demonstrated in a simple example. With a single bad block in a datafile and just 5 reads of the file we end up with a degraded vdev, even though there is only one unique error in the pool. The proposed solution to the above issue, is to eliminate duplicates when posting events and when updating vdev error stats. We now save recent error events of interest when posting events so that we can easily check for duplicates when posting an error. Reviewed by: Brad Lewis <brad.lewis@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Don Brady <don.brady@delphix.com> Closes #10861
2020-09-04 20:34:28 +03:00
uniq
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
vmstat
tests: get rid of which Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Closes #13259
2022-03-09 15:52:14 +03:00
wc'
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
export SYSTEM_FILES_FREEBSD='chflags
compress
ZTS: Misc fixes for FreeBSD * Set geom debug flags in corrupt_blocks_at_level * Use the right time zone for history tests * Add missing commands.cfg entry for diskinfo * Rewrite get_last_txg_synced to use zdb * Don't check ulimits for sparse files * Suspend removal before removing a vdev, not after Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Closes #10054
2020-02-27 20:38:34 +03:00
diskinfo
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
fsck
ZTS: Create xattr helpers to hide platform Create xattr helpers to hide platform and update usage in tests. This does not generally aim to enable all xattr tests yet, but it is a necessary step in that direction. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9826
2020-01-11 00:24:59 +03:00
getextattr
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
gpart
FreeBSD: Fix `zfs jail` and add a test zfs_jail was not using zfs_ioctl so failed to map the IOC number correctly. Use zfs_ioctl to perform the jail ioctl and add a test case for FreeBSD. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Closes #10658
2020-08-01 18:44:54 +03:00
jail
jexec
jls
ZTS: Create xattr helpers to hide platform Create xattr helpers to hide platform and update usage in tests. This does not generally aim to enable all xattr tests yet, but it is a necessary step in that direction. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9826
2020-01-11 00:24:59 +03:00
lsextattr
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
md5
mdconfig
newfs
pw
ZTS: Create xattr helpers to hide platform Create xattr helpers to hide platform and update usage in tests. This does not generally aim to enable all xattr tests yet, but it is a necessary step in that direction. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9826
2020-01-11 00:24:59 +03:00
rmextattr
setextattr
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
sha256
ZTS: Refactor is_shared, fix impl on FreeBSD FreeBSD doesn't have a `share` command. It does have showmount. Split the separate platform impls out of is_shared_impl. Dispatch to the correct platform impl function from is_shared. Eliminate the use of is_shared_impl from tests. is_shared works. Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Closes #10037
2020-02-22 02:59:20 +03:00
showmount
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
swapctl
sysctl
zvol: Support blk-mq for better performance Add support for the kernel's block multiqueue (blk-mq) interface in the zvol block driver. blk-mq creates multiple request queues on different CPUs rather than having a single request queue. This can improve zvol performance with multithreaded reads/writes. This implementation uses the blk-mq interfaces on 4.13 or newer kernels. Building against older kernels will fall back to the older BIO interfaces. Note that you must set the `zvol_use_blk_mq` module param to enable the blk-mq API. It is disabled by default. In addition, this commit lets the zvol blk-mq layer process whole `struct request` IOs at a time, rather than breaking them down into their individual BIOs. This reduces dbuf lock contention and overhead versus the legacy zvol submit_bio() codepath. sequential dd to one zvol, 8k volblocksize, no O_DIRECT: legacy submit_bio() 292MB/s write 453MB/s read this commit 453MB/s write 885MB/s read It also introduces a new `zvol_blk_mq_chunks_per_thread` module parameter. This parameter represents how many volblocksize'd chunks to process per each zvol thread. It can be used to tune your zvols for better read vs write performance (higher values favor write, lower favor read). Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com> Signed-off-by: Tony Hutter <hutter2@llnl.gov> Closes #13148 Issue #12483
2022-06-09 17:10:38 +03:00
trim
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
uncompress'
export SYSTEM_FILES_LINUX='attr
blkid
zvol: Support blk-mq for better performance Add support for the kernel's block multiqueue (blk-mq) interface in the zvol block driver. blk-mq creates multiple request queues on different CPUs rather than having a single request queue. This can improve zvol performance with multithreaded reads/writes. This implementation uses the blk-mq interfaces on 4.13 or newer kernels. Building against older kernels will fall back to the older BIO interfaces. Note that you must set the `zvol_use_blk_mq` module param to enable the blk-mq API. It is disabled by default. In addition, this commit lets the zvol blk-mq layer process whole `struct request` IOs at a time, rather than breaking them down into their individual BIOs. This reduces dbuf lock contention and overhead versus the legacy zvol submit_bio() codepath. sequential dd to one zvol, 8k volblocksize, no O_DIRECT: legacy submit_bio() 292MB/s write 453MB/s read this commit 453MB/s write 885MB/s read It also introduces a new `zvol_blk_mq_chunks_per_thread` module parameter. This parameter represents how many volblocksize'd chunks to process per each zvol thread. It can be used to tune your zvols for better read vs write performance (higher values favor write, lower favor read). Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com> Signed-off-by: Tony Hutter <hutter2@llnl.gov> Closes #13148 Issue #12483
2022-06-09 17:10:38 +03:00
blkdiscard
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
blockdev
chattr
exportfs
fallocate
ZTS: Move free to Linux commands list FreeBSD does not have the free command. This command is only used by Linux in a perf hostinfo function. Move free from the list of common commands to the list of Linux commands. Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Closes #10011
2020-02-18 22:23:41 +03:00
free
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
getfattr
groupadd
groupdel
groupmod
hostid
losetup
lsattr
lsblk
lscpu
lsmod
lsscsi
md5sum
mkswap
modprobe
mpstat
Add Linux namespace delegation support This allows ZFS datasets to be delegated to a user/mount namespace Within that namespace, only the delegated datasets are visible Works very similarly to Zones/Jailes on other ZFS OSes As a user: ``` $ unshare -Um $ zfs list no datasets available $ echo $$ 1234 ``` As root: ``` # zfs list NAME ZONED MOUNTPOINT containers off /containers containers/host off /containers/host containers/host/child off /containers/host/child containers/host/child/gchild off /containers/host/child/gchild containers/unpriv on /unpriv containers/unpriv/child on /unpriv/child containers/unpriv/child/gchild on /unpriv/child/gchild # zfs zone /proc/1234/ns/user containers/unpriv ``` Back to the user namespace: ``` $ zfs list NAME USED AVAIL REFER MOUNTPOINT containers 129M 47.8G 24K /containers containers/unpriv 128M 47.8G 24K /unpriv containers/unpriv/child 128M 47.8G 128M /unpriv/child ``` Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Will Andrews <will.andrews@klarasystems.com> Signed-off-by: Allan Jude <allan@klarasystems.com> Signed-off-by: Mateusz Piotrowski <mateusz.piotrowski@klarasystems.com> Co-authored-by: Allan Jude <allan@klarasystems.com> Co-authored-by: Mateusz Piotrowski <mateusz.piotrowski@klarasystems.com> Sponsored-by: Buddy <https://buddy.works> Closes #12263
2021-02-21 19:19:43 +03:00
nsenter
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
parted
perf
setfattr
sha256sum
udevadm
Add Linux namespace delegation support This allows ZFS datasets to be delegated to a user/mount namespace Within that namespace, only the delegated datasets are visible Works very similarly to Zones/Jailes on other ZFS OSes As a user: ``` $ unshare -Um $ zfs list no datasets available $ echo $$ 1234 ``` As root: ``` # zfs list NAME ZONED MOUNTPOINT containers off /containers containers/host off /containers/host containers/host/child off /containers/host/child containers/host/child/gchild off /containers/host/child/gchild containers/unpriv on /unpriv containers/unpriv/child on /unpriv/child containers/unpriv/child/gchild on /unpriv/child/gchild # zfs zone /proc/1234/ns/user containers/unpriv ``` Back to the user namespace: ``` $ zfs list NAME USED AVAIL REFER MOUNTPOINT containers 129M 47.8G 24K /containers containers/unpriv 128M 47.8G 24K /unpriv containers/unpriv/child 128M 47.8G 128M /unpriv/child ``` Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Will Andrews <will.andrews@klarasystems.com> Signed-off-by: Allan Jude <allan@klarasystems.com> Signed-off-by: Mateusz Piotrowski <mateusz.piotrowski@klarasystems.com> Co-authored-by: Allan Jude <allan@klarasystems.com> Co-authored-by: Mateusz Piotrowski <mateusz.piotrowski@klarasystems.com> Sponsored-by: Buddy <https://buddy.works> Closes #12263
2021-02-21 19:19:43 +03:00
unshare
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
useradd
userdel
tests: clean out unused/single-use/useless commands from the list Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Closes #13259
2022-03-14 03:41:03 +03:00
usermod
flock
logger'
Update ZTS to work on FreeBSD Update the common ZTS scripts and individual test cases as needed in order to allow them to be run on FreeBSD. The high level goal is to provide compatibility wrappers whenever possible to minimize changes to individual test cases. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Signed-off-by: Matt Macy <mmacy@FreeBSD.org> Signed-off-by: Ryan Moeller <ryan@ixsystems.com> Closes #9692
2019-12-18 23:29:43 +03:00
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
export ZFS_FILES='zdb
zfs
zhack
zinject
zpool
ztest
raidz_test
pyzfs: python3 support (build system) Almost all of the Python code in the respository has been updated to be compatibile with Python 2.6, Python 3.4, or newer. The only exceptions are arc_summery3.py which requires Python 3, and pyzfs which requires at least Python 2.7. This allows us to maintain a single version of the code and support most default versions of python. This change does the following: * Sets the default shebang for all Python scripts to python3. If only Python 2 is available, then at install time scripts which are compatible with Python 2 will have their shebangs replaced with /usr/bin/python. This is done for compatibility until Python 2 goes end of life. Since only the installed versions are changed this means Python 3 must be installed on the system for test-runner when testing in-tree. * Added --with-python=<2|3|3.4,etc> configure option which sets the PYTHON environment variable to target a specific python version. By default the newest installed version of Python will be used or the preferred distribution version when creating pacakges. * Fixed --enable-pyzfs configure checks so they are run when --enable-pyzfs=check and --enable-pyzfs=yes. * Enabled pyzfs for Python 3.4 and newer, which is now supported. * Renamed pyzfs package to python<VERSION>-pyzfs and updated to install in the appropriate site location. For example, when building with --with-python=3.4 a python34-pyzfs will be created which installs in /usr/lib/python3.4/site-packages/. * Renamed the following python scripts according to the Fedora guidance for packaging utilities in /bin - dbufstat.py -> dbufstat - arcstat.py -> arcstat - arc_summary.py -> arc_summary - arc_summary3.py -> arc_summary3 * Updated python-cffi package name. On CentOS 6, CentOS 7, and Amazon Linux it's called python-cffi, not python2-cffi. For Python3 it's called python3-cffi or python3x-cffi. * Install one version of arc_summary. Depending on the version of Python available install either arc_summary2 or arc_summary3 as arc_summary. The user output is only slightly different. Reviewed-by: John Ramsden <johnramsden@riseup.net> Reviewed-by: Neal Gompa <ngompa@datto.com> Reviewed-by: loli10K <ezomori.nozomu@gmail.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #8096
2018-10-31 19:22:59 +03:00
arc_summary
arcstat
Add zilstat script to report zil kstats in a user friendly manner Added a python script to process both global and per dataset zil kstats and report them in a user friendly manner similar to arcstat and dbufstat. Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Reviewed-by: Alexander Motin <mav@FreeBSD.org> Reviewed-by: Richard Elling <Richard.Elling@RichardElling.com> Signed-off-by: Ameer Hamza <ahamza@ixsystems.com> Closes #13704
2022-09-02 23:24:07 +03:00
zilstat
pyzfs: python3 support (build system) Almost all of the Python code in the respository has been updated to be compatibile with Python 2.6, Python 3.4, or newer. The only exceptions are arc_summery3.py which requires Python 3, and pyzfs which requires at least Python 2.7. This allows us to maintain a single version of the code and support most default versions of python. This change does the following: * Sets the default shebang for all Python scripts to python3. If only Python 2 is available, then at install time scripts which are compatible with Python 2 will have their shebangs replaced with /usr/bin/python. This is done for compatibility until Python 2 goes end of life. Since only the installed versions are changed this means Python 3 must be installed on the system for test-runner when testing in-tree. * Added --with-python=<2|3|3.4,etc> configure option which sets the PYTHON environment variable to target a specific python version. By default the newest installed version of Python will be used or the preferred distribution version when creating pacakges. * Fixed --enable-pyzfs configure checks so they are run when --enable-pyzfs=check and --enable-pyzfs=yes. * Enabled pyzfs for Python 3.4 and newer, which is now supported. * Renamed pyzfs package to python<VERSION>-pyzfs and updated to install in the appropriate site location. For example, when building with --with-python=3.4 a python34-pyzfs will be created which installs in /usr/lib/python3.4/site-packages/. * Renamed the following python scripts according to the Fedora guidance for packaging utilities in /bin - dbufstat.py -> dbufstat - arcstat.py -> arcstat - arc_summary.py -> arc_summary - arc_summary3.py -> arc_summary3 * Updated python-cffi package name. On CentOS 6, CentOS 7, and Amazon Linux it's called python-cffi, not python2-cffi. For Python3 it's called python3-cffi or python3x-cffi. * Install one version of arc_summary. Depending on the version of Python available install either arc_summary2 or arc_summary3 as arc_summary. The user output is only slightly different. Reviewed-by: John Ramsden <johnramsden@riseup.net> Reviewed-by: Neal Gompa <ngompa@datto.com> Reviewed-by: loli10K <ezomori.nozomu@gmail.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #8096
2018-10-31 19:22:59 +03:00
dbufstat
Fix memleak in cmd/mount_zfs.c Convert dynamic allocation to static buffer, simplify parse_dataset function return path. Add tests specific to the mount helper. Reviewed-by: Mateusz Guzik <mjguzik@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Sterling Jensen <sterlingjensen@users.noreply.github.com> Closes #11098
2020-11-11 02:50:44 +03:00
mount.zfs
OpenZFS 7252 - compressed zfs send / receive OpenZFS 7252 - compressed zfs send / receive OpenZFS 7628 - create long versions of ZFS send / receive options Authored by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Paul Dagnelie <pcd@delphix.com> Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed by: Sebastien Roy <sebastien.roy@delphix.com> Reviewed by: David Quigley <dpquigl@davequigley.com> Reviewed by: Thomas Caputi <tcaputi@datto.com> Approved by: Dan McDonald <danmcd@omniti.com> Reviewed by: David Quigley <dpquigl@davequigley.com> Reviewed-by: loli10K <ezomori.nozomu@gmail.com> Ported-by: bunder2015 <omfgbunder@gmail.com> Ported-by: Don Brady <don.brady@intel.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Porting Notes: - Most of 7252 was already picked up during ABD work. This commit represents the gap from the final commit to openzfs. - Fixed split_large_blocks check in do_dump() - An alternate version of the write_compressible() function was implemented for Linux which does not depend on fio. The behavior of fio differs significantly based on the exact version. - mkholes was replaced with truncate for Linux. OpenZFS-issue: https://www.illumos.org/issues/7252 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/5602294 Closes #6067
2017-04-12 00:56:54 +03:00
zed
Add zgenhostid utility script Turning the multihost property on requires that a hostid be set to allow ZFS to determine when a foreign system is attemping to import a pool. The error message instructing the user to set a hostid refers to genhostid(1). Genhostid(1) is not available on SUSE Linux. This commit adds a script modeled after genhostid(1) for those users. Zgenhostid checks for an /etc/hostid file; if it does not exist, it creates one and stores a value. If the user has provided a hostid as an argument, that value is used. Otherwise, a random hostid is generated and stored. This differs from the CENTOS 6/7 versions of genhostid, which overwrite the /etc/hostid file even though their manpages state otherwise. A man page for zgenhostid is added. The one for genhostid is in (1), but I put zgenhostid in (8) because I believe it's more appropriate. The mmp tests are modified to use zgenhostid to set the hostid instead of using the spl_hostid module parameter. zgenhostid will not replace an existing /etc/hostid file, so new mmp_clear_hostid calls are required. Reviewed-by: Giuseppe Di Natale <dinatale2@llnl.gov> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Olaf Faaland <faaland1@llnl.gov> Closes #6358 Closes #6379
2017-07-19 04:11:08 +03:00
zgenhostid
Add `zstream redup` command to convert deduplicated send streams Deduplicated send and receive is deprecated. To ease migration to the new dedup-send-less world, the commit adds a `zstream redup` utility to convert deduplicated send streams to normal streams, so that they can continue to be received indefinitely. The new `zstream` command also replaces the functionality of `zstreamdump`, by way of the `zstream dump` subcommand. The `zstreamdump` command is replaced by a shell script which invokes `zstream dump`. The way that `zstream redup` works under the hood is that as we read the send stream, we build up a hash table which maps from `<GUID, object, offset> -> <file_offset>`. Whenever we see a WRITE record, we add a new entry to the hash table, which indicates where in the stream file to find the WRITE record for this block. (The key is `drr_toguid, drr_object, drr_offset`.) For entries other than WRITE_BYREF, we pass them through unchanged (except for the running checksum, which is recalculated). For WRITE_BYREF records, we change them to WRITE records. We find the referenced WRITE record by looking in the hash table (for the record with key `drr_refguid, drr_refobject, drr_refoffset`), and then reading the record header and payload from the specified offset in the stream file. This is why the stream can not be a pipe. The found WRITE record replaces the WRITE_BYREF record, with its `drr_toguid`, `drr_object`, and `drr_offset` fields changed to be the same as the WRITE_BYREF's (i.e. we are writing the same logical block, but with the data supplied by the previous WRITE record). This algorithm requires memory proportional to the number of WRITE records (same as `zfs send -D`), but the size per WRITE record is relatively low (40 bytes, vs. 72 for `zfs send -D`). A 1TB send stream with 8KB blocks (`recordsize=8k`) would use around 5GB of RAM to "redup". Reviewed-by: Jorgen Lundman <lundman@lundman.net> Reviewed-by: Paul Dagnelie <pcd@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Matthew Ahrens <mahrens@delphix.com> Closes #10124 Closes #10156
2020-04-10 20:39:55 +03:00
zstream
Add zpool_influxdb command A zpool_influxdb command is introduced to ease the collection of zpool statistics into the InfluxDB time-series database. Examples are given on how to integrate with the telegraf statistics aggregator, a companion to influxdb. Finally, a grafana dashboard template is included to show how pool latency distributions can be visualized in a ZFS + telegraf + influxdb + grafana environment. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Richard Elling <Richard.Elling@RichardElling.com> Closes #10786
2020-10-09 19:29:21 +03:00
zfs_ids_to_path
zpool_influxdb'
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
Drop references when skipping dmu_send due to EXDEV When an invalid incremental send is requested where the "to" ds is before the "from" ds, make sure to drop the reference to the pool and the dataset before returning the error. Add an assert on FreeBSD to make sure we don't hold any locks after returning from an ioctl. Add some test coverage. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Ryan Moeller <ryan@iXsystems.com> Closes #10919
2020-09-30 23:19:49 +03:00
export ZFSTEST_FILES='badsend
btree_test
Tests for btree implementation used by range trees Additional test cases for the btree implementation, see #9181. Reviewed-by: Paul Dagnelie <pcd@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: John Kennedy <john.kennedy@delphix.com> Closes #9717
2019-12-19 22:53:55 +03:00
chg_usr_exec
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
devname2devid
dir_rd_update
Distributed Spare (dRAID) Feature This patch adds a new top-level vdev type called dRAID, which stands for Distributed parity RAID. This pool configuration allows all dRAID vdevs to participate when rebuilding to a distributed hot spare device. This can substantially reduce the total time required to restore full parity to pool with a failed device. A dRAID pool can be created using the new top-level `draid` type. Like `raidz`, the desired redundancy is specified after the type: `draid[1,2,3]`. No additional information is required to create the pool and reasonable default values will be chosen based on the number of child vdevs in the dRAID vdev. zpool create <pool> draid[1,2,3] <vdevs...> Unlike raidz, additional optional dRAID configuration values can be provided as part of the draid type as colon separated values. This allows administrators to fully specify a layout for either performance or capacity reasons. The supported options include: zpool create <pool> \ draid[<parity>][:<data>d][:<children>c][:<spares>s] \ <vdevs...> - draid[parity] - Parity level (default 1) - draid[:<data>d] - Data devices per group (default 8) - draid[:<children>c] - Expected number of child vdevs - draid[:<spares>s] - Distributed hot spares (default 0) Abbreviated example `zpool status` output for a 68 disk dRAID pool with two distributed spares using special allocation classes. ``` pool: tank state: ONLINE config: NAME STATE READ WRITE CKSUM slag7 ONLINE 0 0 0 draid2:8d:68c:2s-0 ONLINE 0 0 0 L0 ONLINE 0 0 0 L1 ONLINE 0 0 0 ... U25 ONLINE 0 0 0 U26 ONLINE 0 0 0 spare-53 ONLINE 0 0 0 U27 ONLINE 0 0 0 draid2-0-0 ONLINE 0 0 0 U28 ONLINE 0 0 0 U29 ONLINE 0 0 0 ... U42 ONLINE 0 0 0 U43 ONLINE 0 0 0 special mirror-1 ONLINE 0 0 0 L5 ONLINE 0 0 0 U5 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 L6 ONLINE 0 0 0 U6 ONLINE 0 0 0 spares draid2-0-0 INUSE currently in use draid2-0-1 AVAIL ``` When adding test coverage for the new dRAID vdev type the following options were added to the ztest command. These options are leverages by zloop.sh to test a wide range of dRAID configurations. -K draid|raidz|random - kind of RAID to test -D <value> - dRAID data drives per group -S <value> - dRAID distributed hot spares -R <value> - RAID parity (raidz or dRAID) The zpool_create, zpool_import, redundancy, replacement and fault test groups have all been updated provide test coverage for the dRAID feature. Co-authored-by: Isaac Huang <he.huang@intel.com> Co-authored-by: Mark Maybee <mmaybee@cray.com> Co-authored-by: Don Brady <don.brady@delphix.com> Co-authored-by: Matthew Ahrens <mahrens@delphix.com> Co-authored-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Mark Maybee <mmaybee@cray.com> Reviewed-by: Matt Ahrens <matt@delphix.com> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #10102
2020-11-14 00:51:51 +03:00
draid
Adding ZTS test for O_APPEND Commit 63b18e4 fixed an issue in zpl_aio_write() to make sure that kiocb->ki_pos was updated correctly when opening a file with O_APPEND. Adding a test to verify O_APPEND functionality with lseek can make sure that all other distros/kernel versions also have the correct behavior. Also moved the threadappends_001_pos test into this append test directory in functional ZTS directory. This way the two append tests are together for organization purposes. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Brian Atkinson <batkinson@lanl.gov> Closes #13424
2022-05-11 18:38:16 +03:00
file_append
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
file_check
file_trunc
file_write
Implement Redacted Send/Receive Redacted send/receive allows users to send subsets of their data to a target system. One possible use case for this feature is to not transmit sensitive information to a data warehousing, test/dev, or analytics environment. Another is to save space by not replicating unimportant data within a given dataset, for example in backup tools like zrepl. Redacted send/receive is a three-stage process. First, a clone (or clones) is made of the snapshot to be sent to the target. In this clone (or clones), all unnecessary or unwanted data is removed or modified. This clone is then snapshotted to create the "redaction snapshot" (or snapshots). Second, the new zfs redact command is used to create a redaction bookmark. The redaction bookmark stores the list of blocks in a snapshot that were modified by the redaction snapshot(s). Finally, the redaction bookmark is passed as a parameter to zfs send. When sending to the snapshot that was redacted, the redaction bookmark is used to filter out blocks that contain sensitive or unwanted information, and those blocks are not included in the send stream. When sending from the redaction bookmark, the blocks it contains are considered as candidate blocks in addition to those blocks in the destination snapshot that were modified since the creation_txg of the redaction bookmark. This step is necessary to allow the target to rehydrate data in the case where some blocks are accidentally or unnecessarily modified in the redaction snapshot. The changes to bookmarks to enable fast space estimation involve adding deadlists to bookmarks. There is also logic to manage the life cycles of these deadlists. The new size estimation process operates in cases where previously an accurate estimate could not be provided. In those cases, a send is performed where no data blocks are read, reducing the runtime significantly and providing a byte-accurate size estimate. Reviewed-by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed-by: Matt Ahrens <mahrens@delphix.com> Reviewed-by: Prashanth Sreenivasa <pks@delphix.com> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: George Wilson <george.wilson@delphix.com> Reviewed-by: Chris Williamson <chris.williamson@delphix.com> Reviewed-by: Pavel Zhakarov <pavel.zakharov@delphix.com> Reviewed-by: Sebastien Roy <sebastien.roy@delphix.com> Reviewed-by: Prakash Surya <prakash.surya@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Paul Dagnelie <pcd@delphix.com> Closes #7958
2019-06-19 19:48:13 +03:00
get_diff
Linux: Implement FS_IOC_GETVERSION Provide access to file generation number on Linux. Add test coverage. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Signed-off-by: Ryan Moeller <freqlabs@FreeBSD.org> Closes #12856
2021-12-18 03:18:37 +03:00
getversion
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
largest_file
Add basic zfs ioc input nvpair validation We want newer versions of libzfs_core to run against an existing zfs kernel module (i.e. a deferred reboot or module reload after an update). Programmatically document, via a zfs_ioc_key_t, the valid arguments for the ioc commands that rely on nvpair input arguments (i.e. non legacy commands from libzfs_core). Automatically verify the expected pairs before dispatching a command. This initial phase focuses on the non-legacy ioctls. A follow-on change can address the legacy ioctl input from the zfs_cmd_t. The zfs_ioc_key_t for zfs_keys_channel_program looks like: static const zfs_ioc_key_t zfs_keys_channel_program[] = { {"program", DATA_TYPE_STRING, 0}, {"arg", DATA_TYPE_UNKNOWN, 0}, {"sync", DATA_TYPE_BOOLEAN_VALUE, ZK_OPTIONAL}, {"instrlimit", DATA_TYPE_UINT64, ZK_OPTIONAL}, {"memlimit", DATA_TYPE_UINT64, ZK_OPTIONAL}, }; Introduce four input errors to identify specific input failures (in addition to generic argument value errors like EINVAL, ERANGE, EBADF, and E2BIG). ZFS_ERR_IOC_CMD_UNAVAIL the ioctl number is not supported by kernel ZFS_ERR_IOC_ARG_UNAVAIL an input argument is not supported by kernel ZFS_ERR_IOC_ARG_REQUIRED a required input argument is missing ZFS_ERR_IOC_ARG_BADTYPE an input argument has an invalid type Reviewed-by: Matthew Ahrens <mahrens@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Don Brady <don.brady@delphix.com> Closes #7780
2018-09-02 22:14:01 +03:00
libzfs_input_check
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
mkbusy
mkfile
mkfiles
mktree
mmap_exec
Fix mmap / libaio deadlock Calling uiomove() in mappedread() under the page lock can result in a deadlock if the user space page needs to be faulted in. Resolve the issue by dropping the page lock before the uiomove(). The inode range lock protects against concurrent updates via zfs_read() and zfs_write(). Reviewed-by: Albert Lee <trisk@forkgnu.org> Reviewed-by: Chunwei Chen <david.chen@nutanix.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes #7335 Closes #7339
2018-03-28 20:19:22 +03:00
mmap_libaio
Fix lseek(SEEK_DATA/SEEK_HOLE) mmap consistency When using lseek(2) to report data/holes memory mapped regions of the file were ignored. This could result in incorrect results. To handle this zfs_holey_common() was updated to asynchronously writeback any dirty mmap(2) regions prior to reporting holes. Additionally, while not strictly required, the dn_struct_rwlock is now held over the dirty check to prevent the dnode structure from changing. This ensures that a clean dnode can't be dirtied before the data/hole is located. The range lock is now also taken to ensure the call cannot race with zfs_write(). Furthermore, the code was refactored to provide a dnode_is_dirty() helper function which checks the dnode for any dirty records to determine its dirtiness. Reviewed-by: Matthew Ahrens <mahrens@delphix.com> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Reviewed-by: Rich Ercolani <rincebrain@gmail.com> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Issue #11900 Closes #12724
2021-11-08 00:27:44 +03:00
mmap_seek
Speed up WB_SYNC_NONE when a WB_SYNC_ALL occurs simultaneously Page writebacks with WB_SYNC_NONE can take several seconds to complete since they wait for the transaction group to close before being committed. This is usually not a problem since the caller does not need to wait. However, if we're simultaneously doing a writeback with WB_SYNC_ALL (e.g via msync), the latter can block for several seconds (up to zfs_txg_timeout) due to the active WB_SYNC_NONE writeback since it needs to wait for the transaction to complete and the PG_writeback bit to be cleared. This commit deals with 2 cases: - No page writeback is active. A WB_SYNC_ALL page writeback starts and even completes. But when it's about to check if the PG_writeback bit has been cleared, another writeback with WB_SYNC_NONE starts. The sync page writeback ends up waiting for the non-sync page writeback to complete. - A page writeback with WB_SYNC_NONE is already active when a WB_SYNC_ALL writeback starts. The WB_SYNC_ALL writeback ends up waiting for the WB_SYNC_NONE writeback. The fix works by carefully keeping track of active sync/non-sync writebacks and committing when beneficial. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Shaan Nobee <sniper111@gmail.com> Closes #12662 Closes #12790
2022-05-03 23:23:26 +03:00
mmap_sync
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
mmapwrite
OpenZFS 7431 - ZFS Channel Programs Authored by: Chris Williamson <chris.williamson@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Approved by: Garrett D'Amore <garrett@damore.org> Ported-by: Don Brady <don.brady@delphix.com> Ported-by: John Kennedy <john.kennedy@delphix.com> OpenZFS-issue: https://www.illumos.org/issues/7431 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/dfc11533 Porting Notes: * The CLI long option arguments for '-t' and '-m' don't parse on linux * Switched from kmem_alloc to vmem_alloc in zcp_lua_alloc * Lua implementation is built as its own module (zlua.ko) * Lua headers consumed directly by zfs code moved to 'include/sys/lua/' * There is no native setjmp/longjump available in stock Linux kernel. Brought over implementations from illumos and FreeBSD * The get_temporary_prop() was adapted due to VFS platform differences * Use of inline functions in lua parser to reduce stack usage per C call * Skip some ZFS Test Suite ZCP tests on sparc64 to avoid stack overflow
2018-02-08 19:16:23 +03:00
nvlist_to_lua
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
randfree_file
OpenZFS 9166 - zfs storage pool checkpoint Details about the motivation of this feature and its usage can be found in this blogpost: https://sdimitro.github.io/post/zpool-checkpoint/ A lightning talk of this feature can be found here: https://www.youtube.com/watch?v=fPQA8K40jAM Implementation details can be found in big block comment of spa_checkpoint.c Side-changes that are relevant to this commit but not explained elsewhere: * renames members of "struct metaslab trees to be shorter without losing meaning * space_map_{alloc,truncate}() accept a block size as a parameter. The reason is that in the current state all space maps that we allocate through the DMU use a global tunable (space_map_blksz) which defauls to 4KB. This is ok for metaslab space maps in terms of bandwirdth since they are scattered all over the disk. But for other space maps this default is probably not what we want. Examples are device removal's vdev_obsolete_sm or vdev_chedkpoint_sm from this review. Both of these have a 1:1 relationship with each vdev and could benefit from a bigger block size. Porting notes: * The part of dsl_scan_sync() which handles async destroys has been moved into the new dsl_process_async_destroys() function. * Remove "VERIFY(!(flags & FWRITE))" in "kernel.c" so zhack can write to block device backed pools. * ZTS: * Fix get_txg() in zpool_sync_001_pos due to "checkpoint_txg". * Don't use large dd block sizes on /dev/urandom under Linux in checkpoint_capacity. * Adopt Delphix-OS's setting of 4 (spa_asize_inflation = SPA_DVAS_PER_BP + 1) for the checkpoint_capacity test to speed its attempts to fill the pool * Create the base and nested pools with sync=disabled to speed up the "setup" phase. * Clear labels in test pool between checkpoint tests to avoid duplicate pool issues. * The import_rewind_device_replaced test has been marked as "known to fail" for the reasons listed in its DISCLAIMER. * New module parameters: zfs_spa_discard_memory_limit, zfs_remove_max_bytes_pause (not documented - debugging only) vdev_max_ms_count (formerly metaslabs_per_vdev) vdev_min_ms_count Authored by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov> Approved by: Richard Lowe <richlowe@richlowe.net> Ported-by: Tim Chase <tim@chase2k.com> Signed-off-by: Tim Chase <tim@chase2k.com> OpenZFS-issue: https://illumos.org/issues/9166 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/7159fdb8 Closes #7570
2016-12-17 01:11:29 +03:00
randwritecomp
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
readmmap
Expose additional file level attributes ZFS allows to update and retrieve additional file level attributes for FreeBSD. This commit allows additional file level attributes to be updated and retrieved for Linux. These include the flags stored in the upper half of z_pflags only. Two new IOCTLs have been added for this purpose. ZFS_IOC_GETDOSFLAGS can be used to retrieve the attributes, while ZFS_IOC_SETDOSFLAGS can be used to update the attributes. Attributes that are allowed to be updated include ZFS_IMMUTABLE, ZFS_APPENDONLY, ZFS_NOUNLINK, ZFS_ARCHIVE, ZFS_NODUMP, ZFS_SYSTEM, ZFS_HIDDEN, ZFS_READONLY, ZFS_REPARSE, ZFS_OFFLINE and ZFS_SPARSE. Flags can be or'd together while calling ZFS_IOC_SETDOSFLAGS. Reviewed-by: Alexander Motin <mav@FreeBSD.org> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Umer Saleem <usaleem@ixsystems.com> Closes #13118
2022-03-08 04:52:03 +03:00
read_dos_attributes
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
rename_dir
rm_lnkcnt_zero_file
send_iterate_snap : doall send without fromsnap The behavior of a NULL fromsnap was inadvertently changed for a doall send when the send/recv logic in libzfs was updated. Restore the previous behavior by correcting send_iterate_snap() to include all the snapshots in the nvlist for this case. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Cedric Maunoury <cedric.maunoury@gmail.com> Closes #11608
2021-02-24 20:48:58 +03:00
send_doall
OpenZFS 7290 - ZFS test suite needs to control what utilities it can run Authored by: John Wren Kennedy <john.kennedy@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@omniti.com> Approved by: Gordon Ross <gordon.w.ross@gmail.com> Ported-by: Brian Behlendorf <behlendorf1@llnl.gov> Ported-by: George Melikov <mail@gmelikov.ru> Porting Notes: - Utilities which aren't available under Linux have been removed. - Because of sudo's default secure path behavior PATH must be explicitly reset at the top of libtest.shlib. This avoids the need for all users to customize secure path on their system. - Updated ZoL infrastructure to manage constrained path - Updated all test cases - Check permissions for usergroup tests - When testing in-tree create links under bin/ - Update fault cleanup such that missing files during cleanup aren't fatal. - Configure su environment with constrained path OpenZFS-issue: https://www.illumos.org/issues/7290 OpenZFS-commit: https://github.com/openzfs/openzfs/commit/1d32ba6 Closes #5903
2017-04-06 03:18:22 +03:00
threadsappend
Take user namespaces into account in policy checks Change file related checks to use user namespaces and make sure involved uids/gids are mappable in the current namespace. Note that checks without file ownership information will still not take user namespaces into account, as some of these should be handled via 'zfs allow' (otherwise root in a user namespace could issue commands such as `zpool export`). This also adds an initial user namespace regression test for the setgid bit loss, with a user_ns_exec helper usable in further tests. Additionally, configure checks for the required user namespace related features are added for: * ns_capable * kuid/kgid_has_mapping() * user_ns in cred_t Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com> Closes #6800 Closes #7270
2018-03-08 02:40:42 +03:00
user_ns_exec
Expose additional file level attributes ZFS allows to update and retrieve additional file level attributes for FreeBSD. This commit allows additional file level attributes to be updated and retrieved for Linux. These include the flags stored in the upper half of z_pflags only. Two new IOCTLs have been added for this purpose. ZFS_IOC_GETDOSFLAGS can be used to retrieve the attributes, while ZFS_IOC_SETDOSFLAGS can be used to update the attributes. Attributes that are allowed to be updated include ZFS_IMMUTABLE, ZFS_APPENDONLY, ZFS_NOUNLINK, ZFS_ARCHIVE, ZFS_NODUMP, ZFS_SYSTEM, ZFS_HIDDEN, ZFS_READONLY, ZFS_REPARSE, ZFS_OFFLINE and ZFS_SPARSE. Flags can be or'd together while calling ZFS_IOC_SETDOSFLAGS. Reviewed-by: Alexander Motin <mav@FreeBSD.org> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Umer Saleem <usaleem@ixsystems.com> Closes #13118
2022-03-08 04:52:03 +03:00
write_dos_attributes
Implement Redacted Send/Receive Redacted send/receive allows users to send subsets of their data to a target system. One possible use case for this feature is to not transmit sensitive information to a data warehousing, test/dev, or analytics environment. Another is to save space by not replicating unimportant data within a given dataset, for example in backup tools like zrepl. Redacted send/receive is a three-stage process. First, a clone (or clones) is made of the snapshot to be sent to the target. In this clone (or clones), all unnecessary or unwanted data is removed or modified. This clone is then snapshotted to create the "redaction snapshot" (or snapshots). Second, the new zfs redact command is used to create a redaction bookmark. The redaction bookmark stores the list of blocks in a snapshot that were modified by the redaction snapshot(s). Finally, the redaction bookmark is passed as a parameter to zfs send. When sending to the snapshot that was redacted, the redaction bookmark is used to filter out blocks that contain sensitive or unwanted information, and those blocks are not included in the send stream. When sending from the redaction bookmark, the blocks it contains are considered as candidate blocks in addition to those blocks in the destination snapshot that were modified since the creation_txg of the redaction bookmark. This step is necessary to allow the target to rehydrate data in the case where some blocks are accidentally or unnecessarily modified in the redaction snapshot. The changes to bookmarks to enable fast space estimation involve adding deadlists to bookmarks. There is also logic to manage the life cycles of these deadlists. The new size estimation process operates in cases where previously an accurate estimate could not be provided. In those cases, a send is performed where no data blocks are read, reducing the runtime significantly and providing a byte-accurate size estimate. Reviewed-by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed-by: Matt Ahrens <mahrens@delphix.com> Reviewed-by: Prashanth Sreenivasa <pks@delphix.com> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: George Wilson <george.wilson@delphix.com> Reviewed-by: Chris Williamson <chris.williamson@delphix.com> Reviewed-by: Pavel Zhakarov <pavel.zakharov@delphix.com> Reviewed-by: Sebastien Roy <sebastien.roy@delphix.com> Reviewed-by: Prakash Surya <prakash.surya@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Paul Dagnelie <pcd@delphix.com> Closes #7958
2019-06-19 19:48:13 +03:00
xattrtest
tests: move C test helpers into test cmd Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Closes #13259
2022-03-22 22:09:35 +03:00
stride_dd
zed_fd_spill-zedlet
suid_write_to_file
cp_files
Introduce BLAKE3 checksums as an OpenZFS feature This commit adds BLAKE3 checksums to OpenZFS, it has similar performance to Edon-R, but without the caveats around the latter. Homepage of BLAKE3: https://github.com/BLAKE3-team/BLAKE3 Wikipedia: https://en.wikipedia.org/wiki/BLAKE_(hash_function)#BLAKE3 Short description of Wikipedia: BLAKE3 is a cryptographic hash function based on Bao and BLAKE2, created by Jack O'Connor, Jean-Philippe Aumasson, Samuel Neves, and Zooko Wilcox-O'Hearn. It was announced on January 9, 2020, at Real World Crypto. BLAKE3 is a single algorithm with many desirable features (parallelism, XOF, KDF, PRF and MAC), in contrast to BLAKE and BLAKE2, which are algorithm families with multiple variants. BLAKE3 has a binary tree structure, so it supports a practically unlimited degree of parallelism (both SIMD and multithreading) given enough input. The official Rust and C implementations are dual-licensed as public domain (CC0) and the Apache License. Along with adding the BLAKE3 hash into the OpenZFS infrastructure a new benchmarking file called chksum_bench was introduced. When read it reports the speed of the available checksum functions. On Linux: cat /proc/spl/kstat/zfs/chksum_bench On FreeBSD: sysctl kstat.zfs.misc.chksum_bench This is an example output of an i3-1005G1 test system with Debian 11: implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 1196 1602 1761 1749 1762 1759 1751 skein-generic 546 591 608 615 619 612 616 sha256-generic 240 300 316 314 304 285 276 sha512-generic 353 441 467 476 472 467 426 blake3-generic 308 313 313 313 312 313 312 blake3-sse2 402 1289 1423 1446 1432 1458 1413 blake3-sse41 427 1470 1625 1704 1679 1607 1629 blake3-avx2 428 1920 3095 3343 3356 3318 3204 blake3-avx512 473 2687 4905 5836 5844 5643 5374 Output on Debian 5.10.0-10-amd64 system: (Ryzen 7 5800X) implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 1840 2458 2665 2719 2711 2723 2693 skein-generic 870 966 996 992 1003 1005 1009 sha256-generic 415 442 453 455 457 457 457 sha512-generic 608 690 711 718 719 720 721 blake3-generic 301 313 311 309 309 310 310 blake3-sse2 343 1865 2124 2188 2180 2181 2186 blake3-sse41 364 2091 2396 2509 2463 2482 2488 blake3-avx2 365 2590 4399 4971 4915 4802 4764 Output on Debian 5.10.0-9-powerpc64le system: (POWER 9) implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 1213 1703 1889 1918 1957 1902 1907 skein-generic 434 492 520 522 511 525 525 sha256-generic 167 183 187 188 188 187 188 sha512-generic 186 216 222 221 225 224 224 blake3-generic 153 152 154 153 151 153 153 blake3-sse2 391 1170 1366 1406 1428 1426 1414 blake3-sse41 352 1049 1212 1174 1262 1258 1259 Output on Debian 5.10.0-11-arm64 system: (Pi400) implementation 1k 4k 16k 64k 256k 1m 4m edonr-generic 487 603 629 639 643 641 641 skein-generic 271 299 303 308 309 309 307 sha256-generic 117 127 128 130 130 129 130 sha512-generic 145 165 170 172 173 174 175 blake3-generic 81 29 71 89 89 89 89 blake3-sse2 112 323 368 379 380 371 374 blake3-sse41 101 315 357 368 369 364 360 Structurally, the new code is mainly split into these parts: - 1x cross platform generic c variant: blake3_generic.c - 4x assembly for X86-64 (SSE2, SSE4.1, AVX2, AVX512) - 2x assembly for ARMv8 (NEON converted from SSE2) - 2x assembly for PPC64-LE (POWER8 converted from SSE2) - one file for switching between the implementations Note the PPC64 assembly requires the VSX instruction set and the kfpu_begin() / kfpu_end() calls on PowerPC were updated accordingly. Reviewed-by: Felix Dörre <felix@dogcraft.de> Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Tino Reichardt <milky-zfs@mcmilk.de> Co-authored-by: Rich Ercolani <rincebrain@gmail.com> Closes #10058 Closes #12918
2022-06-09 01:55:57 +03:00
blake3_test
tests: move C test helpers into test cmd Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: John Kennedy <john.kennedy@delphix.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz> Closes #13259
2022-03-22 22:09:35 +03:00
edonr_test
skein_test
sha2_test
ctime
truncate_test
ereports
zfs_diff-socket
dosmode_readonly_write'
Reference in New Issue Copy Permalink