c/mirror_zfs
1
0
Fork 0
mirror of https://git.proxmox.com/git/mirror_zfs.git synced 2025-03-11 14:03:10 +03:00
Code Issues Packages Projects Releases Wiki Activity
5c474614ff
mirror_zfs/include/sys
History
Attila Fülöp 76354f945e ICP: Improve AES-GCM performance 
Currently SIMD accelerated AES-GCM performance is limited by two
factors:

a. The need to disable preemption and interrupts and save the FPU
state before using it and to do the reverse when done. Due to the
way the code is organized (see (b) below) we have to pay this price
twice for each 16 byte GCM block processed.

b. Most processing is done in C, operating on single GCM blocks.
The use of SIMD instructions is limited to the AES encryption of the
counter block (AES-NI) and the Galois multiplication (PCLMULQDQ).
This leads to the FPU not being fully utilized for crypto
operations.

To solve (a) we do crypto processing in larger chunks while owning
the FPU. An `icp_gcm_avx_chunk_size` module parameter was introduced
to make this chunk size tweakable. It defaults to 32 KiB. This step
alone roughly doubles performance. (b) is tackled by porting and
using the highly optimized openssl AES-GCM assembler routines, which
do all the processing (CTR, AES, GMULT) in a single routine. Both
steps together result in up to 32x reduction of the time spend in
the en/decryption routines, leading up to approximately 12x
throughput increase for large (128 KiB) blocks.

Lastly, this commit changes the default encryption algorithm from
AES-CCM to AES-GCM when setting the `encryption=on` property.

Reviewed-By: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-By: Jason King <jason.king@joyent.com>
Reviewed-By: Tom Caputi <tcaputi@datto.com>
Reviewed-By: Richard Laager <rlaager@wiktel.com>
Signed-off-by: Attila Fülöp <attila@fueloep.org>
Closes #9749
2020-05-12 10:53:32 -07:00
..
crypto Add support for selecting encryption backend 2018-08-02 11:59:24 -07:00
fm Add zpool status -s (slow I/Os) and -p (parseable) 2018-11-08 16:47:24 -08:00
fs Fix typos in include/ 2020-01-22 13:48:59 -08:00
lua Fix typos in include/ 2020-01-22 13:48:59 -08:00
sysevent Add TRIM support 2019-03-29 09:13:20 -07:00
abd.h single-chunk scatter ABDs can be treated as linear 2020-01-22 13:48:56 -08:00
aggsum.h OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
arc_impl.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
arc.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
avl_impl.h
avl.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
blkptr.h
bplist.h
bpobj.h OpenZFS 7614, 9064 - zfs device evacuation/removal 2018-04-14 12:16:17 -07:00
bptree.h
bqueue.h
cityhash.h OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
dataset_kstats.h port async unlinked drain from illumos-nexenta 2019-02-12 10:41:15 -08:00
dbuf.h Linux 4.19-rc3+ compat: Remove refcount_t compat 2018-09-26 10:29:26 -07:00
ddt.h Incorrect maximum DVA value in DDE_GET_NDVAS() 2018-02-26 14:20:12 -08:00
dmu_impl.h Fix send/recv lost spill block 2019-05-07 15:18:44 -07:00
dmu_objset.h Pool allocation classes 2018-09-05 18:33:36 -07:00
dmu_recv.h Fix send/recv lost spill block 2019-05-07 15:18:44 -07:00
dmu_send.h Refactor dmu_recv into its own file 2018-10-09 14:05:13 -07:00
dmu_traverse.h Native Encryption for ZFS on Linux 2017-08-14 10:36:48 -07:00
dmu_tx.h Linux 4.19-rc3+ compat: Remove refcount_t compat 2018-09-26 10:29:26 -07:00
dmu_zfetch.h
dmu.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
dnode.h Fix zil replay panic when TX_REMOVE followed by TX_CREATE 2019-09-25 11:27:51 -07:00
dsl_bookmark.h Fix comments on zfs_bookmark_phys 2019-09-25 11:27:48 -07:00
dsl_crypt.h Allow unencrypted children of encrypted datasets 2019-09-25 11:27:48 -07:00
dsl_dataset.h Detect and prevent mixed raw and non-raw sends 2019-03-13 11:00:43 -07:00
dsl_deadlist.h OpenZFS 7614, 9064 - zfs device evacuation/removal 2018-04-14 12:16:17 -07:00
dsl_deleg.h OpenZFS 7614, 9064 - zfs device evacuation/removal 2018-04-14 12:16:17 -07:00
dsl_destroy.h OpenZFS 7431 - ZFS Channel Programs 2018-02-08 15:28:18 -08:00
dsl_dir.h Remove duplicate macro in dsl_dir.h 2018-10-01 10:40:11 -07:00
dsl_pool.h port async unlinked drain from illumos-nexenta 2019-02-12 10:41:15 -08:00
dsl_prop.h
dsl_scan.h Prevent unnecessary resilver restarts 2020-01-22 13:49:07 -08:00
dsl_synctask.h OpenZFS 9425 - channel programs can be interrupted 2020-01-22 13:48:56 -08:00
dsl_userhold.h
edonr.h
efi_partition.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
frame.h Suppress incorrect objtool warnings 2017-12-07 10:28:50 -08:00
hkdf.h Encryption patch follow-up 2017-10-11 16:54:48 -04:00
Makefile.am Add TRIM support 2019-03-29 09:13:20 -07:00
metaslab_impl.h Add TRIM support 2019-03-29 09:13:20 -07:00
metaslab.h Allow metaslab to be unloaded even when not freed from 2019-09-25 11:27:47 -07:00
mmp.h MMP interval and fail_intervals in uberblock 2019-03-21 12:47:57 -07:00
mntent.h
multilist.h Avoid extra taskq_dispatch() calls by DMU 2019-09-25 11:27:48 -07:00
note.h Update build system and packaging 2018-05-29 16:00:33 -07:00
nvpair_impl.h OpenZFS 9580 - Add a hash-table on top of nvlist to speed-up operations 2018-07-30 11:30:03 -07:00
nvpair.h Add new fnvlist_lookup_* functions 2018-10-03 15:30:55 -07:00
pathname.h Disable unused pathname::pn_path* (unneeded in Linux) 2019-09-25 11:27:49 -07:00
policy.h
range_tree.h Rename range_tree_verify to range_tree_verify_not_present 2019-01-25 09:51:24 -08:00
refcount.h Prevent race in blkptr_verify against device removal 2020-01-22 13:48:57 -08:00
rrwlock.h Linux 4.19-rc3+ compat: Remove refcount_t compat 2018-09-26 10:29:26 -07:00
sa_impl.h Linux 4.19-rc3+ compat: Remove refcount_t compat 2018-09-26 10:29:26 -07:00
sa.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
sdt.h Add line info and SET_ERROR() to ZFS debug log 2017-07-25 23:09:48 -07:00
sha2.h
skein.h
spa_boot.h
spa_checkpoint.h Serialize ZTHR operations to eliminate races 2019-01-13 10:09:46 -08:00
spa_checksum.h
spa_impl.h spa_load_verify() may consume too much memory 2020-01-22 13:48:57 -08:00
spa.h Prevent unnecessary resilver restarts 2020-01-22 13:49:07 -08:00
space_map.h Introduce auxiliary metaslab histograms 2019-02-20 09:59:56 -08:00
space_reftree.h
sysevent.h
trace_acl.h Linux 4.16 compat: inode_set_iversion() 2018-02-08 21:25:19 -08:00
trace_arc.h Support re-prioritizing asynchronous prefetches 2017-12-21 09:13:06 -08:00
trace_common.h
trace_dbgmsg.h Add line info and SET_ERROR() to ZFS debug log 2017-07-25 23:09:48 -07:00
trace_dbuf.h Prefix all refcount functions with zfs_ 2018-10-01 10:42:05 -07:00
trace_dmu.h tx_waited -> tx_dirty_delayed in trace_dmu.h 2018-01-31 16:13:26 -08:00
trace_dnode.h
trace_multilist.h
trace_txg.h
trace_vdev.h OpenZFS 7614, 9064 - zfs device evacuation/removal 2018-04-14 12:16:17 -07:00
trace_zil.h OpenZFS 8585 - improve batching done in zil_commit() 2017-12-05 09:39:16 -08:00
trace_zio.h
trace_zrlock.h Fix race in trace point in zrl_add_impl 2018-03-12 11:27:02 -07:00
trace.h
txg_impl.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
txg.h OpenZFS 9425 - channel programs can be interrupted 2020-01-22 13:48:56 -08:00
u8_textprep_data.h
u8_textprep.h
uberblock_impl.h MMP interval and fail_intervals in uberblock 2019-03-21 12:47:57 -07:00
uberblock.h Multi-modifier protection (MMP) 2017-07-13 13:54:00 -04:00
uio_impl.h deadlock between mm_sem and tx assign in zfs_write() and page fault 2018-10-16 11:11:24 -07:00
unique.h
uuid.h
vdev_disk.h Add support for autoexpand property 2018-07-23 15:40:15 -07:00
vdev_file.h
vdev_impl.h Linux 5.6 compat: time_t 2020-05-12 10:53:32 -07:00
vdev_indirect_births.h OpenZFS 7614, 9064 - zfs device evacuation/removal 2018-04-14 12:16:17 -07:00
vdev_indirect_mapping.h OpenZFS 7614, 9064 - zfs device evacuation/removal 2018-04-14 12:16:17 -07:00
vdev_initialize.h Add TRIM support 2019-03-29 09:13:20 -07:00
vdev_raidz_impl.h Linux 5.0 compat: SIMD compatibility 2020-01-22 13:49:01 -08:00
vdev_raidz.h Linux 5.0 compat: SIMD compatibility 2020-01-22 13:49:01 -08:00
vdev_removal.h panic in removal_remap test on 4K devices 2019-09-25 11:27:47 -07:00
vdev_trim.h Add TRIM support 2019-03-29 09:13:20 -07:00
vdev.h Prevent unnecessary resilver restarts 2020-01-22 13:49:07 -08:00
xvattr.h Linux 4.18 compat: inode timespec -> timespec64 2018-06-19 21:51:18 -07:00
zap_impl.h
zap_leaf.h Fix ENOSPC in "Handle zap_add() failures in ..." 2018-04-18 14:19:50 -07:00
zap.h fat zap should prefetch when iterating 2019-09-25 11:27:47 -07:00
zcp_global.h OpenZFS 7431 - ZFS Channel Programs 2018-02-08 15:28:18 -08:00
zcp_iter.h OpenZFS 7431 - ZFS Channel Programs 2018-02-08 15:28:18 -08:00
zcp_prop.h OpenZFS 7431 - ZFS Channel Programs 2018-02-08 15:28:18 -08:00
zcp.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
zfeature.h
zfs_acl.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
zfs_context.h OpenZFS 9425 - channel programs can be interrupted 2020-01-22 13:48:56 -08:00
zfs_ctldir.h
zfs_debug.h Add TRIM support 2019-03-29 09:13:20 -07:00
zfs_delay.h Update build system and packaging 2018-05-29 16:00:33 -07:00
zfs_dir.h port async unlinked drain from illumos-nexenta 2019-02-12 10:41:15 -08:00
zfs_fuid.h Update build system and packaging 2018-05-29 16:00:33 -07:00
zfs_ioctl.h Fix send/recv lost spill block 2019-05-07 15:18:44 -07:00
zfs_onexit.h
zfs_project.h Project Quota on ZFS 2018-02-13 14:54:54 -08:00
zfs_ratelimit.h Change checksum & IO delay ratelimit values 2018-03-04 17:34:51 -08:00
zfs_rlock.h Rename rangelock_ functions to zfs_rangelock_ 2020-01-22 13:49:02 -08:00
zfs_sa.h Project Quota on ZFS 2018-02-13 14:54:54 -08:00
zfs_stat.h
zfs_sysfs.h Fix in-kernel sysfs entries 2018-09-06 21:44:52 -07:00
zfs_vfsops.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
zfs_vnops.h RHEL 7.5 compat: FMODE_KABI_ITERATE 2018-05-02 15:01:24 -07:00
zfs_znode.h Change boolean-like uint8_t fields in znode_t to boolean_t 2020-01-22 13:48:57 -08:00
zil_impl.h make zil max block size tunable 2020-01-22 13:48:56 -08:00
zil.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
zio_checksum.h
zio_compress.h lz4_decompress_abd declared but not defined 2019-09-25 11:27:47 -07:00
zio_crypt.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
zio_impl.h Fix typos in include/ 2020-01-22 13:48:59 -08:00
zio_priority.h Add TRIM support 2019-03-29 09:13:20 -07:00
zio.h ICP: Improve AES-GCM performance 2020-05-12 10:53:32 -07:00
zpl.h Linux 5.6 compat: timestamp_truncate() 2020-05-12 10:53:32 -07:00
zrlock.h
zthr.h Serialize ZTHR operations to eliminate races 2019-01-13 10:09:46 -08:00
zvol.h