Romain Dolbeau 62a65a654e Add parity generation/rebuild using 128-bits NEON for Aarch64 ... This re-use the framework established for SSE2, SSSE3 and AVX2. However, GCC is using FP registers on Aarch64, so unlike SSE/AVX2 we can't rely on the registers being left alone between ASM statements. So instead, the NEON code uses C variables and GCC extended ASM syntax. Note that since the kernel explicitly disable vector registers, they have to be locally re-enabled explicitly. As we use the variable's number to define the symbolic name, and GCC won't allow duplicate symbolic names, numbers have to be unique. Even when the code is not going to be used (e.g. the case for 4 registers when using the macro with only 2). Only the actually used variables should be declared, otherwise the build will fails in debug mode. This requires the replacement of the XOR(X,X) syntax by a new ZERO(X) macro, which does the same thing but without repeating the argument. And perhaps someday there will be a machine where there is a more efficient way to zero a register than XOR with itself. This affects scalar, SSE2, SSSE3 and AVX2 as they need the new macro. It's possible to write faster implementations (different scheduling, different unrolling, interleaving NEON and scalar, ...) for various cores, but this one has the advantage of fitting in the current state of the code, and thus is likely easier to review/check/merge. The only difference between aarch64-neon and aarch64-neonx2 is that aarch64-neonx2 unroll some functions some more. Reviewed-by: Gvozden Neskovic <neskovic@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Romain Dolbeau <romain.dolbeau@atos.net> Closes #4801		2016-10-03 09:44:00 -07:00
..
arc.c	Fix cppcheck warning in buf_init()	2016-09-30 15:04:21 -07:00
blkptr.c	Illumos 4757, 4913	2014-08-01 14:28:05 -07:00
bplist.c	Change KM_PUSHPAGE -> KM_SLEEP	2015-01-16 14:41:26 -08:00
bpobj.c	Illumos 5810 - zdb should print details of bpobj	2015-05-11 15:10:24 -07:00
bptree.c	Illumos 5960, 5925	2016-01-08 15:08:19 -08:00
bqueue.c	Allow 16M send/recv blocks	2016-01-08 20:23:23 -05:00
dbuf_stats.c	OpenZFS 6950 - ARC should cache compressed data	2016-09-13 09:58:33 -07:00
dbuf.c	fix: Shift exponent too large	2016-09-29 15:55:41 -07:00
ddt_zap.c	Change KM_PUSHPAGE -> KM_SLEEP	2015-01-16 14:41:26 -08:00
ddt.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
dmu_diff.c	OpenZFS 6950 - ARC should cache compressed data	2016-09-13 09:58:33 -07:00
dmu_object.c	Implement large_dnode pool feature	2016-06-24 13:13:21 -07:00
dmu_objset.c	DLPX-40252 integrate EP-476 compressed zfs send/receive	2016-09-13 09:58:58 -07:00
dmu_send.c	OpenZFS 7230 - add assertions to dmu_send_impl() to verify that stream includes BEGIN and END records	2016-09-22 16:01:19 -07:00
dmu_traverse.c	Enable ignore_hole_birth module option by default	2016-09-16 14:05:30 -07:00
dmu_tx.c	OpenZFS 7004 - dmu_tx_hold_zap() does dnode_hold() 7x on same object	2016-08-19 12:48:03 -07:00
dmu_zfetch.c	OpenZFS 6322 - ZFS indirect block predictive prefetch	2016-08-30 14:26:55 -07:00
dmu.c	DLPX-44733 combine arc_buf_alloc_impl() with arc_buf_clone()	2016-09-13 09:59:13 -07:00
dnode_sync.c	OpenZFS 6950 - ARC should cache compressed data	2016-09-13 09:58:33 -07:00
dnode.c	fix: Shift exponent too large	2016-09-29 15:55:41 -07:00
dsl_bookmark.c	OpenZFS 6314 - buffer overflow in dsl_dataset_name	2016-06-28 13:47:03 -07:00
dsl_dataset.c	Fix coverity defects: CID 147563, 147560	2016-09-30 15:56:17 -07:00
dsl_deadlist.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
dsl_deleg.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
dsl_destroy.c	OpenZFS 2605, 6980, 6902	2016-06-28 13:47:02 -07:00
dsl_dir.c	Fix coverity defects: 147658, 147652, 147651	2016-09-29 12:06:14 -07:00
dsl_pool.c	Fix self-healing IO prior to dsl_pool_init() completion	2016-05-27 14:11:25 -07:00
dsl_prop.c	Free property names with spa_strfree() rather than strfree()	2016-09-12 09:45:26 -07:00
dsl_scan.c	OpenZFS 6950 - ARC should cache compressed data	2016-09-13 09:58:33 -07:00
dsl_synctask.c	Illumos 4951 - ZFS administrative commands should use reserved space	2015-05-04 09:41:10 -07:00
dsl_userhold.c	OpenZFS 6314 - buffer overflow in dsl_dataset_name	2016-06-28 13:47:03 -07:00
fm.c	Fix coverity defects: CID 147448, 147449, 147450, 147453, 147454	2016-10-02 11:24:54 -07:00
gzip.c	cstyle: Resolve C style issues	2013-12-18 16:46:35 -08:00
lz4.c	Change KM_PUSHPAGE -> KM_SLEEP	2015-01-16 14:41:26 -08:00
lzjb.c	Change KM_PUSHPAGE -> KM_SLEEP	2015-01-16 14:41:26 -08:00
Makefile.in	Add parity generation/rebuild using 128-bits NEON for Aarch64	2016-10-03 09:44:00 -07:00
metaslab.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
multilist.c	Identify locks flagged by lockdep	2015-12-22 10:21:33 -08:00
pathname.c	Add pn_alloc()/pn_free() functions	2016-04-21 09:49:25 -07:00
policy.c	Fix NFS credential	2016-06-21 09:58:37 -07:00
range_tree.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
refcount.c	OpenZFS 6950 - ARC should cache compressed data	2016-09-13 09:58:33 -07:00
rrwlock.c	Illumos 5008 - lock contention (rrw_exit) while running a read only load	2015-07-06 09:34:13 -07:00
sa.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
sha256.c	Explicit integer promotion for bit shift operations	2016-09-29 15:55:41 -07:00
spa_boot.c	Add linux kernel module support	2010-08-31 13:41:58 -07:00
spa_config.c	OpenZFS 5997 - FRU field not set during pool creation and never updated	2016-08-12 13:06:48 -07:00
spa_errlog.c	Illumos 4914 - zfs on-disk bookmark structure should be named *_phys_t	2014-08-06 14:48:41 -07:00
spa_history.c	Fix indefinite article	2016-08-11 11:23:49 -07:00
spa_misc.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
spa_stats.c	Explicit integer promotion for bit shift operations	2016-09-29 15:55:41 -07:00
spa.c	Fix coverity defects	2016-09-21 18:09:00 -07:00
space_map.c	Illumos 5960, 5925	2016-01-08 15:08:19 -08:00
space_reftree.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
trace.c	OpenZFS 6531 - Provide mechanism to artificially limit disk performance	2016-05-26 10:11:51 -07:00
txg.c	txg visibility code should not execute under tc_open_lock	2016-07-27 14:11:13 -07:00
uberblock.c	Illumos 5347 - idle pool may run itself out of space	2015-07-14 10:35:21 -07:00
unique.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
vdev_cache.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
vdev_disk.c	Explicit block device plugging when submitting multiple BIOs	2016-09-29 13:13:31 -07:00
vdev_file.c	OpenZFS 6531 - Provide mechanism to artificially limit disk performance	2016-05-26 10:11:51 -07:00
vdev_label.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
vdev_mirror.c	FreeBSD r256956: Improve ZFS N-way mirror read performance by using load and locality information.	2016-02-26 11:24:35 -08:00
vdev_missing.c	Illumos #5244 - zio pipeline callers should explicitly invoke next stage	2015-04-30 15:07:47 -07:00
vdev_queue.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
vdev_raidz_math_aarch64_neon_common.h	Add parity generation/rebuild using 128-bits NEON for Aarch64	2016-10-03 09:44:00 -07:00
vdev_raidz_math_aarch64_neon.c	Add parity generation/rebuild using 128-bits NEON for Aarch64	2016-10-03 09:44:00 -07:00
vdev_raidz_math_aarch64_neonx2.c	Add parity generation/rebuild using 128-bits NEON for Aarch64	2016-10-03 09:44:00 -07:00
vdev_raidz_math_avx2.c	Add parity generation/rebuild using 128-bits NEON for Aarch64	2016-10-03 09:44:00 -07:00
vdev_raidz_math_impl.h	Add parity generation/rebuild using 128-bits NEON for Aarch64	2016-10-03 09:44:00 -07:00
vdev_raidz_math_scalar.c	Add parity generation/rebuild using 128-bits NEON for Aarch64	2016-10-03 09:44:00 -07:00
vdev_raidz_math_sse2.c	Add parity generation/rebuild using 128-bits NEON for Aarch64	2016-10-03 09:44:00 -07:00
vdev_raidz_math_ssse3.c	Add parity generation/rebuild using 128-bits NEON for Aarch64	2016-10-03 09:44:00 -07:00
vdev_raidz_math.c	Add parity generation/rebuild using 128-bits NEON for Aarch64	2016-10-03 09:44:00 -07:00
vdev_raidz.c	Build user-space with different gcc optimization levels	2016-08-09 14:40:35 -07:00
vdev_root.c	Illumos #3598	2013-10-31 14:58:04 -07:00
vdev.c	Bring over illumos ZFS FMA logic -- phase 1	2016-09-01 11:39:45 -07:00
zap_leaf.c	Illumos 5314 - Remove "dbuf phys" db->db_data pointer aliases in ZFS	2015-04-28 16:25:20 -07:00
zap_micro.c	Fix coverity defects: CID 147650, 147649, 147647, 147646	2016-09-25 15:08:28 -07:00
zap.c	Avoid undefined shift overflow in fzap_cursor_retrieve()	2016-09-29 15:55:41 -07:00
zfeature_common.c	Implement large_dnode pool feature	2016-06-24 13:13:21 -07:00
zfeature.c	Revert "zhack: Add 'feature disable' command"	2016-05-17 11:52:07 -07:00
zfs_acl.c	Refactor inode->i_mode management	2016-09-27 14:08:52 -07:00
zfs_byteswap.c	Add linux kernel module support	2010-08-31 13:41:58 -07:00
zfs_ctldir.c	Remove znode's z_uid/z_gid member	2016-07-25 13:21:49 -07:00
zfs_debug.c	Add dbgmsg kstat	2015-09-04 16:08:14 -07:00
zfs_dir.c	Remove znode's z_uid/z_gid member	2016-07-25 13:21:49 -07:00
zfs_fm.c	Bring over illumos ZFS FMA logic -- phase 1	2016-09-01 11:39:45 -07:00
zfs_fuid.c	Fix coverity defects	2016-09-21 18:09:00 -07:00
zfs_ioctl.c	Fix coverity defects: CID 147650, 147649, 147647, 147646	2016-09-25 15:08:28 -07:00
zfs_log.c	Remove znode's z_uid/z_gid member	2016-07-25 13:21:49 -07:00
zfs_onexit.c	zfsdev_getminor() should check for invalid file handles	2015-06-22 17:02:13 -07:00
zfs_replay.c	Fix coverity defects	2016-09-21 18:09:00 -07:00
zfs_rlock.c	Performance optimization of AVL tree comparator functions	2016-08-31 14:35:34 -07:00
zfs_sa.c	Use native inode->i_nlink instead of znode->z_links	2016-07-14 16:25:34 -07:00
zfs_vfsops.c	Remove znode's z_uid/z_gid member	2016-07-25 13:21:49 -07:00
zfs_vnops.c	Refactor inode->i_mode management	2016-09-27 14:08:52 -07:00
zfs_znode.c	Fix coverity defects: CID 147448, 147449, 147450, 147453, 147454	2016-10-02 11:24:54 -07:00
zil.c	OpenZFS 6950 - ARC should cache compressed data	2016-09-13 09:58:33 -07:00
zio_checksum.c	OpenZFS 6950 - ARC should cache compressed data	2016-09-13 09:58:33 -07:00
zio_compress.c	Illumos 5661 - ZFS: "compression = on" should use lz4 if feature is enabled	2015-07-10 12:11:45 -07:00
zio_inject.c	OpenZFS 6531 - Provide mechanism to artificially limit disk performance	2016-05-26 10:11:51 -07:00
zio.c	Enable raw writes to perform dedup with verification	2016-09-13 09:59:04 -07:00
zle.c	Update core ZFS code from build 121 to build 141.	2010-05-28 13:45:14 -07:00
zpl_ctldir.c	Use file_dentry and file_inode wrappers	2016-08-11 12:06:37 -07:00
zpl_export.c	zfsctl: No need to sync ctldir inodes	2015-08-31 13:54:39 -07:00
zpl_file.c	Use file_dentry and file_inode wrappers	2016-08-11 12:06:37 -07:00
zpl_inode.c	Linux 4.7 compat: Fix deadlock during lookup on case-insensitive	2016-09-22 19:09:16 -07:00
zpl_super.c	Fix memleak in zpl_parse_options	2016-05-31 16:04:26 -07:00
zpl_xattr.c	Linux 4.7 compat: fix zpl_get_acl returns invalid acl pointer	2016-08-09 10:03:04 -07:00
zrlock.c	Illumos 5812 - assertion failed in zrl_tryenter(): zr_owner==NULL	2015-04-30 14:43:40 -07:00
zvol.c	Linux 4.8 compat: Fix removal of bio->bi_rw member	2016-08-11 11:19:34 -07:00