mirror_zfs/module
Rich Ercolani f375b23c02
Tiered early abort, zstd edition
It turns out that "do LZ4 and zstd-1 both fail" is a great heuristic
for "don't even bother trying higher zstd tiers".

By way of illustration:
$ cat /incompress | mbuffer | zfs recv -o compression=zstd-12 evenfaster/lowcomp_1M_zstd12_normal
summary: 39.8 GiByte in  3min 40.2sec - average of  185 MiB/s
$ echo 3 | sudo tee /sys/module/zzstd/parameters/zstd_lz4_pass
3
$ cat /incompress | mbuffer -m 4G | zfs recv -o compression=zstd-12 evenfaster/lowcomp_1M_zstd12_patched
summary: 39.8 GiByte in 48.6sec - average of  839 MiB/s
$ sudo zfs list -p -o name,used,lused,ratio evenfaster/lowcomp_1M_zstd12_normal evenfaster/lowcomp_1M_zstd12_patched
NAME                                         USED        LUSED  RATIO
evenfaster/lowcomp_1M_zstd12_normal   39549931520  42721221632   1.08
evenfaster/lowcomp_1M_zstd12_patched  39626399744  42721217536   1.07
$ python3 -c "print(39626399744 - 39549931520)"
76468224
$

I'll take 76 MB out of 42 GB for > 4x speedup.

Reviewed-by: Allan Jude <allan@klarasystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: George Melikov <mail@gmelikov.ru>
Reviewed-by: Kjeld Schouten <kjeld@schouten-lebbing.nl>
Reviewed-by: Ahelenia Ziemiańska <nabijaczleweli@nabijaczleweli.xyz>
Signed-off-by: Rich Ercolani <rincebrain@gmail.com>
Closes #13244
2022-05-24 09:43:22 -07:00
..
avl linux: module: weld all but spl.ko into zfs.ko 2022-04-20 13:28:24 -07:00
icp libzfs: pool: fix false-positives -Wmaybe-uninitialised 2022-05-10 10:18:06 -07:00
lua linux: module: weld all but spl.ko into zfs.ko 2022-04-20 13:28:24 -07:00
nvpair linux: module: weld all but spl.ko into zfs.ko 2022-04-20 13:28:24 -07:00
os linux: spl: generic: ddi_strto*: match solaris ddi_strto*(9) 2022-05-13 10:15:47 -07:00
unicode linux: module: weld all but spl.ko into zfs.ko 2022-04-20 13:28:24 -07:00
zcommon Improve zpool status output, list all affected datasets 2022-04-25 17:25:42 -07:00
zfs Tiered early abort, zstd edition 2022-05-24 09:43:22 -07:00
zstd Tiered early abort, zstd edition 2022-05-24 09:43:22 -07:00
.gitignore Cleanup linux module kbuild files 2020-06-10 09:24:15 -07:00
Kbuild.in Added a workaround for Linux KASAN builds 2022-05-11 13:26:55 -07:00
Makefile.bsd Fix ACL checks for NFS kernel server 2022-03-18 06:47:57 -06:00
Makefile.in autoconf: use include directives instead of recursing down lib 2022-05-10 10:18:11 -07:00