mirror_zfs/include
Rich Ercolani f375b23c02
Tiered early abort, zstd edition
It turns out that "do LZ4 and zstd-1 both fail" is a great heuristic
for "don't even bother trying higher zstd tiers".

By way of illustration:
$ cat /incompress | mbuffer | zfs recv -o compression=zstd-12 evenfaster/lowcomp_1M_zstd12_normal
summary: 39.8 GiByte in  3min 40.2sec - average of  185 MiB/s
$ echo 3 | sudo tee /sys/module/zzstd/parameters/zstd_lz4_pass
3
$ cat /incompress | mbuffer -m 4G | zfs recv -o compression=zstd-12 evenfaster/lowcomp_1M_zstd12_patched
summary: 39.8 GiByte in 48.6sec - average of  839 MiB/s
$ sudo zfs list -p -o name,used,lused,ratio evenfaster/lowcomp_1M_zstd12_normal evenfaster/lowcomp_1M_zstd12_patched
NAME                                         USED        LUSED  RATIO
evenfaster/lowcomp_1M_zstd12_normal   39549931520  42721221632   1.08
evenfaster/lowcomp_1M_zstd12_patched  39626399744  42721217536   1.07
$ python3 -c "print(39626399744 - 39549931520)"
76468224
$

I'll take 76 MB out of 42 GB for > 4x speedup.

Reviewed-by: Allan Jude <allan@klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Kjeld Schouten <kjeld@schouten-lebbing.nl>
Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #13244
2022-05-24 09:43:22 -07:00
..
os Remove hw_serial, ddi_strtoul() 2022-05-13 10:15:31 -07:00
sys Tiered early abort, zstd edition 2022-05-24 09:43:22 -07:00
.gitignore OpenZFS restructuring - move platform specific sources 2019-09-06 11:26:26 -07:00
cityhash.h libzfs: convert to -fvisibility=hidden 2021-06-03 13:17:55 -07:00
libnvpair.h lib{efi,avl,share,tpool,zfs_core,zfsbootenv,zutil}: -fvisibility=hidden 2021-06-09 17:04:32 -07:00
libuutil_common.h Support custom build directories and move includes 2010-09-08 12:38:56 -07:00
libuutil_impl.h Replace /*PRINTFLIKEn*/ with attribute(printf) 2021-07-26 12:07:15 -07:00
libuutil.h Switch from _Noreturn to __attribute__((noreturn)) 2022-03-23 08:51:00 -07:00
libzfs_core.h libzfs, libzfs_core: send: always write to pipe 2022-03-08 09:33:08 -08:00
libzfs.h libzfs: return (allocated) strings instead of filling buffers 2022-05-18 12:52:10 -07:00
libzfsbootenv.h lib{efi,avl,share,tpool,zfs_core,zfsbootenv,zutil}: -fvisibility=hidden 2021-06-09 17:04:32 -07:00
libzutil.h libzfs: constify zfs_strip_partition(), zfs_strip_path() 2022-05-16 15:56:53 -07:00
Makefile.am autoconf: single-step includes 2022-05-10 10:18:51 -07:00
thread_pool.h libspl/include: remove unused/empty headers 2022-05-18 12:10:43 -07:00
zfeature_common.h Improve zpool status output, list all affected datasets 2022-04-25 17:25:42 -07:00
zfs_comutil.h module/*.ko: prune .data, global .rodata 2022-01-14 15:37:55 -08:00
zfs_deleg.h module/*.ko: prune .data, global .rodata 2022-01-14 15:37:55 -08:00
zfs_fletcher.h Add --enable-asan and --enable-ubsan switches 2022-02-03 14:35:38 -08:00
zfs_namecheck.h libzfs: convert to -fvisibility=hidden 2021-06-03 13:17:55 -07:00
zfs_prop.h Flex non-pretty-printed properties and raw-/pretty-print remaining ones 2022-03-04 12:08:33 -08:00