c/mirror_zfs
1
0
Fork 0
mirror of https://git.proxmox.com/git/mirror_zfs.git synced 2024-12-26 03:09:34 +03:00
Code Issues Packages Projects Releases Wiki Activity
9f3f80c0cc
mirror_zfs/module/zfs
History
Brian Atkinson b4e4cbeb20
Always validate checksums for Direct I/O reads 
This fixes an oversight in the Direct I/O PR. There is nothing that
stops a process from manipulating the contents of a buffer for a
Direct I/O read while the I/O is in flight. This can lead checksum
verify failures. However, the disk contents are still correct, and this
would lead to false reporting of checksum validation failures.

To remedy this, all Direct I/O reads that have a checksum verification
failure are treated as suspicious. In the event a checksum validation
failure occurs for a Direct I/O read, then the I/O request will be
reissued though the ARC. This allows for actual validation to happen and
removes any possibility of the buffer being manipulated after the I/O
has been issued.

Just as with Direct I/O write checksum validation failures, Direct I/O
read checksum validation failures are reported though zpool status -d in
the DIO column. Also the zevent has been updated to have both:
1. dio_verify_wr -> Checksum verification failure for writes
2. dio_verify_rd -> Checksum verification failure for reads.
This allows for determining what I/O operation was the culprit for the
checksum verification failure. All DIO errors are reported only on the
top-level VDEV.

Even though FreeBSD can write protect pages (stable pages) it still has
the same issue as Linux with Direct I/O reads.

This commit updates the following:
1. Propogates checksum failures for reads all the way up to the
   top-level VDEV.
2. Reports errors through zpool status -d as DIO.
3. Has two zevents for checksum verify errors with Direct I/O. One for
   read and one for write.
4. Updates FreeBSD ABD code to also check for ABD_FLAG_FROM_PAGES and
   handle ABD buffer contents validation the same as Linux.
5. Updated manipulate_user_buffer.c to also manipulate a buffer while a
   Direct I/O read is taking place.
6. Adds a new ZTS test case dio_read_verify that stress tests the new
   code.
7. Updated man pages.
8. Added an IMPLY statement to zio_checksum_verify() to make sure that
   Direct I/O reads are not issued as speculative.
9. Removed self healing through mirror, raidz, and dRAID VDEVs for
   Direct I/O reads.

This issue was first observed when installing a Windows 11 VM on a ZFS
dataset with the dataset property direct set to always. The zpool
devices would report checksum failures, but running a subsequent zpool
scrub would not repair any data and report no errors.

Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Signed-off-by: Brian Atkinson <batkinson@lanl.gov>
Closes #16598
2024-10-09 12:28:08 -07:00
..
abd.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
aggsum.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
arc.c ARC: Cache arc_c value during arc_evict() 2024-10-04 10:56:43 -07:00
blake3_zfs.c blake3: fix up bogus checksums in face of cpu migration 2023-05-01 17:21:27 -07:00
blkptr.c compress: change zio_compress API to use ABDs 2024-08-22 16:22:24 -07:00
bplist.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
bpobj.c Provide macros for setting and getting blkptr birth times 2024-03-25 15:01:54 -07:00
bptree.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
bqueue.c Batch enqueue/dequeue for bqueue 2023-01-10 13:39:22 -08:00
brt.c BRT: Skip getting length in brt_entry_lookup() 2024-03-25 17:13:45 -07:00
btree.c Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN. 2024-05-10 08:47:21 -07:00
dataset_kstats.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
dbuf_stats.c Consider dnode_t allocations in dbuf cache size accounting 2023-11-17 13:25:53 -08:00
dbuf.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
ddt_log.c Add DDT prune command 2024-09-04 14:17:02 -07:00
ddt_stats.c ddt: dedup log 2024-08-16 12:03:35 -07:00
ddt_zap.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
ddt.c Remove set but not used variable in ddt.c (#16522) 2024-09-10 12:46:50 -07:00
dmu_diff.c Simplify issig(). 2024-05-29 10:49:11 -07:00
dmu_direct.c Always validate checksums for Direct I/O reads 2024-10-09 12:28:08 -07:00
dmu_object.c Replace P2ALIGN with P2ALIGN_TYPED and delete P2ALIGN. 2024-05-10 08:47:21 -07:00
dmu_objset.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dmu_recv.c feature: large_microzap 2024-10-02 20:47:11 -07:00
dmu_redact.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dmu_send.c feature: large_microzap 2024-10-02 20:47:11 -07:00
dmu_traverse.c Provide macros for setting and getting blkptr birth times 2024-03-25 15:01:54 -07:00
dmu_tx.c feature: large_microzap 2024-10-02 20:47:11 -07:00
dmu_zfetch.c Small fix to prefetch ranges aggregation 2024-04-09 16:14:04 -07:00
dmu.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
dnode_sync.c For db_marker inherit the db pointer for AVL comparision. 2023-12-11 14:42:06 -08:00
dnode.c Linux: Report reclaimable memory to kernel as such (#16385) 2024-07-30 11:40:47 -07:00
dsl_bookmark.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_crypt.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_dataset.c Fix generation of kernel uevents for snapshot rename on linux 2024-10-06 14:36:33 -07:00
dsl_deadlist.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_deleg.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_destroy.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_dir.c Support for longnames for files/directories (Linux part) 2024-10-01 13:40:27 -07:00
dsl_pool.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
dsl_prop.c snapdir: add 'disabled' value to make .zfs inaccessible 2024-10-02 09:12:02 -07:00
dsl_scan.c Defer resilver only when progress is above a threshold 2024-10-04 10:41:17 -07:00
dsl_synctask.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
dsl_userhold.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
edonr_zfs.c Remove unused Edon-R variants 2023-03-14 15:59:58 -07:00
fm.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
gzip.c compress: change compression providers API to use ABDs 2024-08-22 16:22:24 -07:00
hkdf.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
lz4_zfs.c compress: change compression providers API to use ABDs 2024-08-22 16:22:24 -07:00
lz4.c lz4: Cherrypick fix for CVE-2021-3520 2022-01-12 16:14:36 -08:00
lzjb.c compress: change compression providers API to use ABDs 2024-08-22 16:22:24 -07:00
metaslab.c Remove extra newline from spa_set_allocator(). 2024-09-17 13:15:42 -07:00
mmp.c vdev probe to slow disk can stall mmp write checker 2024-04-29 14:35:53 -07:00
multilist.c L2ARC: Relax locking during write 2024-04-09 16:23:19 -07:00
objlist.c Implement Redacted Send/Receive 2019-06-19 09:48:12 -07:00
pathname.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
range_tree.c btree: Implement faster binary search algorithm 2023-05-26 10:03:12 -07:00
refcount.c Switch refcount tracking from lists to AVL-trees. 2023-06-14 08:02:27 -07:00
rrwlock.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
sa.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
sha2_zfs.c Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
skein_zfs.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
spa_checkpoint.c RAID-Z expansion feature 2023-11-08 10:19:41 -08:00
spa_config.c Add mutex_enter_interruptible() for interruptible sleeping IOCTLs 2023-10-26 09:17:40 -07:00
spa_errlog.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
spa_history.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
spa_log_spacemap.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
spa_misc.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
spa_stats.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
spa.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
space_map.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
space_reftree.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
THIRDPARTYLICENSE.cityhash OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
THIRDPARTYLICENSE.cityhash.descrip OpenZFS 8484 - Implement aggregate sum and use for arc counters 2018-06-06 09:35:59 -07:00
txg.c vdev probe to slow disk can stall mmp write checker 2024-04-29 14:35:53 -07:00
uberblock.c Provide macros for setting and getting blkptr birth times 2024-03-25 15:01:54 -07:00
unique.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_draid_rand.c Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
vdev_draid.c Always validate checksums for Direct I/O reads 2024-10-09 12:28:08 -07:00
vdev_indirect_births.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
vdev_indirect_mapping.c Remove bcopy(), bzero(), bcmp() 2022-03-15 15:13:42 -07:00
vdev_indirect.c Always validate checksums for Direct I/O reads 2024-10-09 12:28:08 -07:00
vdev_initialize.c Add support for parallel pool exports 2024-05-14 08:57:41 -07:00
vdev_label.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
vdev_mirror.c Always validate checksums for Direct I/O reads 2024-10-09 12:28:08 -07:00
vdev_missing.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_queue.c Fix accounting error for pending sync IO ops in zpool iostat 2023-11-07 09:06:14 -08:00
vdev_raidz_math_aarch64_neon_common.h Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_aarch64_neon.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_aarch64_neonx2.c Fix Clang 15 compilation errors 2022-11-30 13:46:26 -08:00
vdev_raidz_math_avx2.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_avx512bw.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_avx512f.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_impl.h RAIDZ: Use cache blocking during parity math 2023-10-30 14:54:27 -07:00
vdev_raidz_math_powerpc_altivec_common.h Linux ppc64le ieee128 compat: Do not redefine __asm on external headers 2023-01-13 10:58:58 -08:00
vdev_raidz_math_powerpc_altivec.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_scalar.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_sse2.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math_ssse3.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_raidz_math.c Miscellaneous FreBSD compilation bugfixes 2023-04-06 10:35:02 -07:00
vdev_raidz.c Always validate checksums for Direct I/O reads 2024-10-09 12:28:08 -07:00
vdev_rebuild.c Add support for parallel pool exports 2024-05-14 08:57:41 -07:00
vdev_removal.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
vdev_root.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
vdev_trim.c Add support for parallel pool exports 2024-05-14 08:57:41 -07:00
vdev.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
zap_leaf.c ZAP: Some cleanups/micro-optimizations 2024-03-21 16:43:53 -07:00
zap_micro.c feature: large_microzap 2024-10-02 20:47:11 -07:00
zap.c Support for longnames for files/directories (Linux part) 2024-10-01 13:40:27 -07:00
zcp_get.c nvpair: Constify string functions 2023-03-14 15:25:50 -07:00
zcp_global.c OpenZFS 8600 - ZFS channel programs - snapshot 2018-02-08 15:29:24 -08:00
zcp_iter.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
zcp_set.c Support setting user properties in a channel program 2020-02-14 13:41:42 -08:00
zcp_synctask.c Add zfs.sync.snapshot_rename 2022-09-02 13:31:19 -07:00
zcp.c Simplify issig(). 2024-05-29 10:49:11 -07:00
zfeature.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
zfs_byteswap.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_chksum.c Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
zfs_fm.c Adding Direct IO Support 2024-09-14 13:47:59 -07:00
zfs_fuid.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
zfs_impl.c Add generic implementation handling and SHA2 impl 2023-03-02 13:52:21 -08:00
zfs_ioctl.c Support for longnames for files/directories (Linux part) 2024-10-01 13:40:27 -07:00
zfs_log.c zfs_log: add flex array fields to log record structs 2024-09-27 09:18:11 -07:00
zfs_onexit.c Use list_remove_head() where possible. 2023-06-09 10:12:52 -07:00
zfs_quota.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00
zfs_ratelimit.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_replay.c zfs_log: add flex array fields to log record structs 2024-09-27 09:18:11 -07:00
zfs_rlock.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_sa.c Replace dead opensolaris.org license link 2022-07-11 14:16:13 -07:00
zfs_vnops.c Always validate checksums for Direct I/O reads 2024-10-09 12:28:08 -07:00
zfs_znode.c Support for longnames for files/directories (Linux part) 2024-10-01 13:40:27 -07:00
zil.c zfs: add bounds checking to zil_parse (#16308) 2024-07-31 17:17:04 -07:00
zio_checksum.c Provide macros for setting and getting blkptr birth times 2024-03-25 15:01:54 -07:00
zio_compress.c zio_compress: introduce max size threshold 2024-09-19 17:23:58 -07:00
zio_inject.c vdev probe to slow disk can stall mmp write checker 2024-04-29 14:35:53 -07:00
zio.c Always validate checksums for Direct I/O reads 2024-10-09 12:28:08 -07:00
zle.c compress: change compression providers API to use ABDs 2024-08-22 16:22:24 -07:00
zrlock.c Micro-optimize zrl_remove() 2022-11-29 09:26:03 -08:00
zthr.c Switch from _Noreturn to __attribute__((noreturn)) 2022-03-23 08:51:00 -07:00
zvol.c Allocate zap_attribute_t from kmem instead of stack 2024-10-01 13:39:08 -07:00