mirror_zfs/module/zfs
Gvozden Neskovic ab9f4b0b82 SIMD implementation of vdev_raidz generate and reconstruct routines
This is a new implementation of RAIDZ1/2/3 routines using x86_64
scalar, SSE, and AVX2 instruction sets. Included are 3 parity
generation routines (P, PQ, and PQR) and 7 reconstruction routines,
for all RAIDZ level. On module load, a quick benchmark of supported
routines will select the fastest for each operation and they will
be used at runtime. Original implementation is still present and
can be selected via module parameter.

Patch contains:
- specialized gen/rec routines for all RAIDZ levels,
- new scalar raidz implementation (unrolled),
- two x86_64 SIMD implementations (SSE and AVX2 instructions sets),
- fastest routines selected on module load (benchmark).
- cmd/raidz_test - verify and benchmark all implementations
- added raidz_test to the ZFS Test Suite

New zfs module parameters:
- zfs_vdev_raidz_impl (str): selects the implementation to use. On
  module load, the parameter will only accept first 3 options, and
  the other implementations can be set once module is finished
  loading. Possible values for this option are:
    "fastest" - use the fastest math available
    "original" - use the original raidz code
    "scalar" - new scalar impl
    "sse" - new SSE impl if available
    "avx2" - new AVX2 impl if available

See contents of `/sys/module/zfs/parameters/zfs_vdev_raidz_impl` to
get the list of supported values. If an implementation is not supported
on the system, it will not be shown. Currently selected option is
enclosed in `[]`.

Signed-off-by: Gvozden Neskovic <neskovic@gmail.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #4328
2016-06-21 09:27:26 -07:00
..
arc.c Fix arc_prune_task use-after-free 2016-05-25 14:11:53 -07:00
blkptr.c Illumos 4757, 4913 2014-08-01 14:28:05 -07:00
bplist.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
bpobj.c Illumos 5810 - zdb should print details of bpobj 2015-05-11 15:10:24 -07:00
bptree.c Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
bqueue.c Allow 16M send/recv blocks 2016-01-08 20:23:23 -05:00
dbuf_stats.c Illumos 5497 - lock contention on arcs_mtx 2015-06-11 10:27:25 -07:00
dbuf.c Illumos 6844 - dnode_next_offset can detect fictional holes 2016-04-27 16:24:15 -07:00
ddt_zap.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
ddt.c Handle zap_lookup() failure in ddt_object_load() 2015-08-19 14:32:50 -07:00
dmu_diff.c Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
dmu_object.c Illumos 6370 - ZFS send fails to transmit some holes 2016-03-10 14:25:22 -08:00
dmu_objset.c Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
dmu_send.c Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
dmu_traverse.c Illumos 6370 - ZFS send fails to transmit some holes 2016-03-10 14:25:22 -08:00
dmu_tx.c Illumos 4950 - files sometimes can't be removed from a full filesystem 2016-01-21 16:59:30 -08:00
dmu_zfetch.c Illumos 6281 - prefetching should apply to 1MB reads 2016-01-12 13:51:27 -08:00
dmu.c Illumos 4950 - files sometimes can't be removed from a full filesystem 2016-01-21 16:59:30 -08:00
dnode_sync.c Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
dnode.c Illumos 5987 - zfs prefetch code needs work 2016-01-12 09:02:33 -08:00
dsl_bookmark.c Illumos 4951 - ZFS administrative commands should use reserved space 2015-05-04 09:41:10 -07:00
dsl_dataset.c Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
dsl_deadlist.c Handle damaged blk_birth in dsl_deadlist_insert() 2015-12-15 16:12:31 -08:00
dsl_deleg.c Illumos 4951 - ZFS administrative commands should use reserved space 2015-05-04 09:41:10 -07:00
dsl_destroy.c Add support for asynchronous zvol minor operations 2016-03-10 09:49:22 -08:00
dsl_dir.c Add support for asynchronous zvol minor operations 2016-03-10 09:49:22 -08:00
dsl_pool.c Fix self-healing IO prior to dsl_pool_init() completion 2016-05-27 14:11:25 -07:00
dsl_prop.c Illumos 6681 - zfs list burning lots of time in dodefault() via dsl_prop_* 2016-03-15 18:46:44 -07:00
dsl_scan.c Illumos 6537 - Panic on zpool scrub with DEBUG kernel 2016-02-05 11:29:32 -08:00
dsl_synctask.c Illumos 4951 - ZFS administrative commands should use reserved space 2015-05-04 09:41:10 -07:00
dsl_userhold.c Illumos 4951 - ZFS administrative commands should use reserved space 2015-05-04 09:41:10 -07:00
fm.c Illumos 5045 - use atomic_{inc,dec}_* instead of atomic_add_* 2016-01-15 15:38:36 -08:00
gzip.c cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
lz4.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
lzjb.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
Makefile.in SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
metaslab.c gcc build error: -Wbool-compare in metaslab.c 2016-03-30 09:36:51 -07:00
multilist.c Identify locks flagged by lockdep 2015-12-22 10:21:33 -08:00
pathname.c Add pn_alloc()/pn_free() functions 2016-04-21 09:49:25 -07:00
policy.c Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
range_tree.c Illumos 5163 - arc should reap range_seg_cache 2015-06-25 08:58:16 -07:00
refcount.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
rrwlock.c Illumos 5008 - lock contention (rrw_exit) while running a read only load 2015-07-06 09:34:13 -07:00
sa.c Prevent SA length overflow 2015-12-30 13:20:12 -08:00
sha256.c Add linux sha2 support 2010-08-31 13:41:59 -07:00
spa_boot.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
spa_config.c OpenZFS 6736 - ZFS per-vdev ZAPs 2016-05-02 14:27:45 -07:00
spa_errlog.c Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t 2014-08-06 14:48:41 -07:00
spa_history.c Illumos 5027 - zfs large block support 2015-05-11 12:23:16 -07:00
spa_misc.c SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
spa_stats.c Illumos 5369 - arc flags should be an enum 2015-06-11 10:27:25 -07:00
spa.c Add -lhHpw options to "zpool iostat" for avg latency, histograms, & queues 2016-05-12 12:36:32 -07:00
space_map.c Illumos 5960, 5925 2016-01-08 15:08:19 -08:00
space_reftree.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
trace.c OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
txg.c Increase default user space stack size 2016-01-13 13:55:12 -08:00
uberblock.c Illumos 5347 - idle pool may run itself out of space 2015-07-14 10:35:21 -07:00
unique.c Change KM_PUSHPAGE -> KM_SLEEP 2015-01-16 14:41:26 -08:00
vdev_cache.c Illumos 5045 - use atomic_{inc,dec}_* instead of atomic_add_* 2016-01-15 15:38:36 -08:00
vdev_disk.c OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
vdev_file.c OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
vdev_label.c Fix memleak in vdev_config_generate_stats 2016-05-31 16:05:21 -07:00
vdev_mirror.c FreeBSD r256956: Improve ZFS N-way mirror read performance by using load and locality information. 2016-02-26 11:24:35 -08:00
vdev_missing.c Illumos #5244 - zio pipeline callers should explicitly invoke next stage 2015-04-30 15:07:47 -07:00
vdev_queue.c Fix self-healing IO prior to dsl_pool_init() completion 2016-05-27 14:11:25 -07:00
vdev_raidz_math_avx2.c SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
vdev_raidz_math_impl.h SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
vdev_raidz_math_scalar.c SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
vdev_raidz_math_sse.c SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
vdev_raidz_math.c SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
vdev_raidz.c SIMD implementation of vdev_raidz generate and reconstruct routines 2016-06-21 09:27:26 -07:00
vdev_root.c Illumos #3598 2013-10-31 14:58:04 -07:00
vdev.c Add request size histograms (-r) to zpool iostat, minor man page fix 2016-05-25 15:49:35 -07:00
zap_leaf.c Illumos 5314 - Remove "dbuf phys" db->db_data pointer aliases in ZFS 2015-04-28 16:25:20 -07:00
zap_micro.c OpenZFS 6842 - Fix empty xattr dir causing lockup 2016-05-10 10:38:21 -07:00
zap.c OpenZFS 6842 - Fix empty xattr dir causing lockup 2016-05-10 10:38:21 -07:00
zfeature_common.c Illumos 5959 - clean up per-dataset feature count code 2015-12-04 14:20:20 -08:00
zfeature.c Revert "zhack: Add 'feature disable' command" 2016-05-17 11:52:07 -07:00
zfs_acl.c Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
zfs_byteswap.c Add linux kernel module support 2010-08-31 13:41:58 -07:00
zfs_ctldir.c Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
zfs_debug.c Add dbgmsg kstat 2015-09-04 16:08:14 -07:00
zfs_dir.c Fix atime handling and relatime 2016-04-05 18:54:55 -07:00
zfs_fm.c Remove wrong ASSERT in annotate_ecksum 2016-02-17 10:43:02 -08:00
zfs_fuid.c Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
zfs_ioctl.c Add zfs allow and zfs unallow support 2016-06-07 09:16:52 -07:00
zfs_log.c Illumos 5027 - zfs large block support 2015-05-11 12:23:16 -07:00
zfs_onexit.c zfsdev_getminor() should check for invalid file handles 2015-06-22 17:02:13 -07:00
zfs_replay.c Add pn_alloc()/pn_free() functions 2016-04-21 09:49:25 -07:00
zfs_rlock.c Remove dummy znode from zvol_state 2016-05-17 10:29:02 -07:00
zfs_sa.c Kill znode->z_gen field 2016-05-19 13:06:14 -07:00
zfs_vfsops.c Linux 4.6 compat: Fall back to d_prune_aliases() if necessary 2016-06-17 13:33:49 -07:00
zfs_vnops.c Fix out-of-bound access in zfs_fillpage 2016-05-31 16:01:27 -07:00
zfs_znode.c Skip ctldir znode in zfs_rezget to fix snapdir issues 2016-05-23 11:06:56 -07:00
zil.c Align thread priority with Linux defaults 2015-07-28 13:36:47 -07:00
zio_checksum.c Illumos 4757, 4913 2014-08-01 14:28:05 -07:00
zio_compress.c Illumos 5661 - ZFS: "compression = on" should use lz4 if feature is enabled 2015-07-10 12:11:45 -07:00
zio_inject.c OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
zio.c OpenZFS 6531 - Provide mechanism to artificially limit disk performance 2016-05-26 10:11:51 -07:00
zle.c Update core ZFS code from build 121 to build 141. 2010-05-28 13:45:14 -07:00
zpl_ctldir.c Linux 4.7 compat: use iterate_shared for concurrent readdir 2016-05-20 11:09:16 -07:00
zpl_export.c zfsctl: No need to sync ctldir inodes 2015-08-31 13:54:39 -07:00
zpl_file.c Linux 4.7 compat: use iterate_shared for concurrent readdir 2016-05-20 11:09:16 -07:00
zpl_inode.c Add pn_alloc()/pn_free() functions 2016-04-21 09:49:25 -07:00
zpl_super.c Fix memleak in zpl_parse_options 2016-05-31 16:04:26 -07:00
zpl_xattr.c Linux 4.5 compat: Use xattr_handler->name for acl 2016-04-25 08:42:08 -07:00
zrlock.c Illumos 5812 - assertion failed in zrl_tryenter(): zr_owner==NULL 2015-04-30 14:43:40 -07:00
zvol.c Linux 4.7 compat: replace blk_queue_flush with blk_queue_write_cache 2016-05-20 11:08:55 -07:00