mirror_zfs/lib/libspl/include/sys
Attila Fülöp 31b160f0a6
ICP: Improve AES-GCM performance
Currently SIMD accelerated AES-GCM performance is limited by two
factors:

a. The need to disable preemption and interrupts and save the FPU
state before using it and to do the reverse when done. Due to the
way the code is organized (see (b) below) we have to pay this price
twice for each 16 byte GCM block processed.

b. Most processing is done in C, operating on single GCM blocks.
The use of SIMD instructions is limited to the AES encryption of the
counter block (AES-NI) and the Galois multiplication (PCLMULQDQ).
This leads to the FPU not being fully utilized for crypto
operations.

To solve (a) we do crypto processing in larger chunks while owning
the FPU. An `icp_gcm_avx_chunk_size` module parameter was introduced
to make this chunk size tweakable. It defaults to 32 KiB. This step
alone roughly doubles performance. (b) is tackled by porting and
using the highly optimized openssl AES-GCM assembler routines, which
do all the processing (CTR, AES, GMULT) in a single routine. Both
steps together result in up to 32x reduction of the time spend in
the en/decryption routines, leading up to approximately 12x
throughput increase for large (128 KiB) blocks.

Lastly, this commit changes the default encryption algorithm from
AES-CCM to AES-GCM when setting the `encryption=on` property.

Reviewed-By: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-By: Jason King <jason.king@joyent.com>
Reviewed-By: Tom Caputi <tcaputi@datto.com>
Reviewed-By: Richard Laager <rlaager@wiktel.com>
Signed-off-by: Attila Fülöp <attila@fueloep.org>
Closes #9749
2020-02-10 12:59:50 -08:00
..
dktp Remove autotools products 2012-08-27 11:47:44 -07:00
acl_impl.h Add linux libspl support 2010-08-31 13:41:59 -07:00
acl.h Add linux libspl support 2010-08-31 13:41:59 -07:00
callb.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
cmn_err.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
cred.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
debug.h Replace ASSERTV macro with compiler annotation 2019-12-05 12:37:00 -08:00
dkio.h Fix spelling 2017-01-03 11:31:18 -06:00
dklabel.h Fix spelling 2017-01-03 11:31:18 -06:00
feature_tests.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
int_limits.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
int_types.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
inttypes.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
isa_defs.h Preliminary support for RV64G 2019-11-06 10:56:09 -08:00
kmem.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
kstat.h Fix typos in lib/ 2019-09-02 17:53:27 -07:00
list_impl.h Add linux libspl support 2010-08-31 13:41:59 -07:00
list.h Add linux libspl support 2010-08-31 13:41:59 -07:00
Makefile.am Refactor zfs_context.h to build on FreeBSD 2019-12-04 13:12:57 -08:00
mhd.h Add linux libspl support 2010-08-31 13:41:59 -07:00
mkdev.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
policy.h Illumos 3897 - zfs filesystem and snapshot limits 2015-04-28 16:22:51 -07:00
poll.h Fix header inclusions for standards conformance 2017-04-12 08:58:09 -07:00
priv.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
processor.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
sha2.h Move sha2.h to platform code 2019-10-31 15:45:58 -07:00
simd.h ICP: Improve AES-GCM performance 2020-02-10 12:59:50 -08:00
stack.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
stdtypes.h Update build system and packaging 2018-05-29 16:00:33 -07:00
strings.h Update build system and packaging 2018-05-29 16:00:33 -07:00
stropts.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
sunddi.h Add linux libspl support 2010-08-31 13:41:59 -07:00
systeminfo.h Add libtpool (thread pools) 2017-08-09 15:31:08 -07:00
time.h Linux 4.18 compat: inode timespec -> timespec64 2018-06-19 21:51:18 -07:00
trace_spl.h Enable use of DTRACE_PROBE* macros in "spl" module 2019-11-01 13:13:43 -07:00
trace_zfs.h Enable use of DTRACE_PROBE* macros in "spl" module 2019-11-01 13:13:43 -07:00
types32.h Update build system and packaging 2018-05-29 16:00:33 -07:00
types.h Update build system and packaging 2018-05-29 16:00:33 -07:00
tzfile.h Add linux libspl support 2010-08-31 13:41:59 -07:00
va_list.h Replace __va_list with va_list 2014-08-13 10:35:00 -07:00
varargs.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
vnode.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00
vtoc.h Fix typos in lib/ 2019-09-02 17:53:27 -07:00
zone.h cstyle: Resolve C style issues 2013-12-18 16:46:35 -08:00