Alexander Motin ed2f7ba08d Implement uncached prefetch ... Previously the primarycache property was handled only in the dbuf layer. Since the speculative prefetcher is implemented in the ARC, it had to be disabled for uncacheable buffers. This change gives the ARC knowledge about uncacheable buffers via arc_read() and arc_write(). So when remove_reference() drops the last reference on the ARC header, it can either immediately destroy it, or if it is marked as prefetch, put it into a new arc_uncached state. That state is scanned every second, evicting stale buffers that were not demand read. This change also tracks dbufs that were read from the beginning, but not to the end. It is assumed that such buffers may receive further reads, and so they are stored in dbuf cache. If a following reads reaches the end of the buffer, it is immediately evicted. Otherwise it will follow regular dbuf cache eviction. Since the dbuf layer does not know actual file sizes, this logic is not applied to the final buffer of a dnode. Since uncacheable buffers should no longer stay in the ARC for long, this patch also tries to optimize I/O by allocating ARC physical buffers as linear to allow buffer sharing. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: George Wilson <george.wilson@delphix.com> Reviewed-by: Ryan Moeller <ryan@iXsystems.com> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored by: iXsystems, Inc. Closes #14243		2023-01-04 17:29:54 -07:00
..
abd.c	abd_return_buf() should call zfs_refcount_remove_many() early	2022-10-19 17:11:01 -07:00
aggsum.c
arc.c	Implement uncached prefetch	2023-01-04 17:29:54 -07:00
blake3_zfs.c
blkptr.c
bplist.c
bpobj.c
bptree.c
bqueue.c
btree.c	Optimize microzaps	2022-10-20 11:57:15 -07:00
dataset_kstats.c	Introduce kmem_scnprintf()	2022-10-29 13:05:11 -07:00
dbuf_stats.c
dbuf.c	Implement uncached prefetch	2023-01-04 17:29:54 -07:00
ddt_zap.c
ddt.c
dmu_diff.c
dmu_object.c	Cleanup: Specify unsignedness on things that should not be signed	2022-09-27 16:42:41 -07:00
dmu_objset.c	Implement uncached prefetch	2023-01-04 17:29:54 -07:00
dmu_recv.c	Deny receiving into encrypted datasets if the keys are not loaded	2022-11-03 09:55:13 -07:00
dmu_redact.c
dmu_send.c	Zero end of embedded block buffer in dump_write_embedded()	2022-12-13 17:31:47 -08:00
dmu_traverse.c	arc_read()/arc_access() refactoring and cleanup	2022-12-22 12:10:24 -08:00
dmu_tx.c	Implement uncached prefetch	2023-01-04 17:29:54 -07:00
dmu_zfetch.c	arc_read()/arc_access() refactoring and cleanup	2022-12-22 12:10:24 -08:00
dmu.c	Implement uncached prefetch	2023-01-04 17:29:54 -07:00
dnode_sync.c	Cache dbuf_hash() calculation	2022-12-13 17:29:21 -08:00
dnode.c	Remove few pointer dereferences in dbuf_read()	2022-11-29 09:49:02 -08:00
dsl_bookmark.c	Cleanup: Address Clang's static analyzer's unused code complaints	2022-10-14 13:37:54 -07:00
dsl_crypt.c	Handle and detect #13709's unlock regression (#14161)	2022-11-15 14:44:12 -08:00
dsl_dataset.c	Remove duplicate statically allocated variable	2022-12-08 13:52:42 -08:00
dsl_deadlist.c	Cleanup: 64-bit kernel module parameters should use fixed width types	2022-10-13 10:03:29 -07:00
dsl_deleg.c
dsl_destroy.c
dsl_dir.c	Fix potential NULL pointer dereference regression	2022-11-10 13:56:28 -08:00
dsl_pool.c	Cleanup: Address Clang's static analyzer's unused code complaints	2022-10-14 13:37:54 -07:00
dsl_prop.c	dsl_prop_known_index(): check for invalid prop	2022-11-08 10:16:01 -08:00
dsl_scan.c	deadlock between spa_errlog_lock and dp_config_rwlock	2022-12-22 11:48:49 -08:00
dsl_synctask.c
dsl_userhold.c
edonr_zfs.c
fm.c	fm_fmri_hc_create() must call va_end() before returning	2022-10-18 15:34:36 -07:00
gzip.c
hkdf.c
lz4_zfs.c
lz4.c
lzjb.c
metaslab.c	Bypass metaslab throttle for removal allocations	2022-12-09 10:48:33 -08:00
mmp.c	Cleanup: Address Clang's static analyzer's unused code complaints	2022-10-14 13:37:54 -07:00
multilist.c	Cleanup: Specify unsignedness on things that should not be signed	2022-09-27 16:42:41 -07:00
objlist.c
pathname.c
range_tree.c	Add defensive assertions	2022-10-12 11:25:18 -07:00
refcount.c	Cleanup: Specify unsignedness on things that should not be signed	2022-09-27 16:42:41 -07:00
rrwlock.c
sa.c	Fix double const qualifier declarations	2022-09-30 15:34:39 -07:00
sha256.c
skein_zfs.c
spa_checkpoint.c	Cleanup: 64-bit kernel module parameters should use fixed width types	2022-10-13 10:03:29 -07:00
spa_config.c	zed: mark disks as REMOVED when they are removed	2022-09-28 09:48:46 -07:00
spa_errlog.c	deadlock between spa_errlog_lock and dp_config_rwlock	2022-12-22 11:48:49 -08:00
spa_history.c
spa_log_spacemap.c	Address warnings about possible division by zero from clangsa	2022-11-03 09:58:14 -07:00
spa_misc.c	zed: post a udev change event from spa_vdev_attach()	2022-11-18 11:39:59 -08:00
spa_stats.c	Cleanup: Specify unsignedness on things that should not be signed	2022-09-27 16:42:41 -07:00
spa.c	deadlock between spa_errlog_lock and dp_config_rwlock	2022-12-22 11:48:49 -08:00
space_map.c
space_reftree.c
THIRDPARTYLICENSE.cityhash
THIRDPARTYLICENSE.cityhash.descrip
txg.c	Fix the last two CFI callback prototype mismatches	2022-11-29 09:56:16 -08:00
uberblock.c
unique.c
vdev_cache.c	Cleanup: Specify unsignedness on things that should not be signed	2022-09-27 16:42:41 -07:00
vdev_draid_rand.c
vdev_draid.c
vdev_indirect_births.c
vdev_indirect_mapping.c
vdev_indirect.c	Bump checksum error counter before reporting to ZED	2022-12-02 17:42:22 -08:00
vdev_initialize.c	Cleanup: 64-bit kernel module parameters should use fixed width types	2022-10-13 10:03:29 -07:00
vdev_label.c
vdev_mirror.c
vdev_missing.c
vdev_queue.c	Convert enum zio_flag to uint64_t	2022-10-27 09:54:54 -07:00
vdev_raidz_math_aarch64_neon_common.h
vdev_raidz_math_aarch64_neon.c
vdev_raidz_math_aarch64_neonx2.c	Fix Clang 15 compilation errors	2022-11-30 13:46:26 -08:00
vdev_raidz_math_avx2.c
vdev_raidz_math_avx512bw.c
vdev_raidz_math_avx512f.c
vdev_raidz_math_impl.h
vdev_raidz_math_powerpc_altivec_common.h
vdev_raidz_math_powerpc_altivec.c
vdev_raidz_math_scalar.c
vdev_raidz_math_sse2.c
vdev_raidz_math_ssse3.c
vdev_raidz_math.c	Convert some sprintf() calls to kmem_scnprintf()	2022-11-28 13:49:58 -08:00
vdev_raidz.c	Bump checksum error counter before reporting to ZED	2022-12-02 17:42:22 -08:00
vdev_rebuild.c	Fix sequential resilver drive failure race condition	2022-10-19 15:48:13 -07:00
vdev_removal.c	Bypass metaslab throttle for removal allocations	2022-12-09 10:48:33 -08:00
vdev_root.c
vdev_trim.c	Propagate extent_bytes change to autotrim thread	2022-10-28 10:16:31 -07:00
vdev.c	zed: unclean disk attachment faults the vdev	2022-11-29 09:24:10 -08:00
zap_leaf.c	Optimize microzaps	2022-10-20 11:57:15 -07:00
zap_micro.c	Optimize microzaps	2022-10-20 11:57:15 -07:00
zap.c
zcp_get.c	Cleanup: Address Clang's static analyzer's unused code complaints	2022-10-14 13:37:54 -07:00
zcp_global.c
zcp_iter.c
zcp_set.c
zcp_synctask.c
zcp.c	Fix too few arguments to formatting function	2022-10-29 13:04:52 -07:00
zfeature.c
zfs_byteswap.c
zfs_chksum.c	Introduce kmem_scnprintf()	2022-10-29 13:05:11 -07:00
zfs_fm.c	Remove an unused variable	2022-11-03 10:17:17 -07:00
zfs_fuid.c
zfs_ioctl.c	deadlock between spa_errlog_lock and dp_config_rwlock	2022-12-22 11:48:49 -08:00
zfs_log.c	zfs_rename: support RENAME_* flags	2022-10-28 09:49:20 -07:00
zfs_onexit.c	zfs_onexit_add_cb: make action_handle point to a uintptr_t	2022-11-03 09:52:12 -07:00
zfs_quota.c
zfs_ratelimit.c
zfs_replay.c	Support idmapped mount in user namespace	2022-11-08 10:28:56 -08:00
zfs_rlock.c
zfs_sa.c
zfs_vnops.c	Support idmapped mount in user namespace	2022-11-08 10:28:56 -08:00
zil.c	Optionally skip zil_close during zvol_create_minor_impl	2022-11-08 12:38:08 -08:00
zio_checksum.c	Fix double const qualifier declarations	2022-09-30 15:34:39 -07:00
zio_compress.c	Fix declarations of non-global variables	2022-10-18 11:05:32 -07:00
zio_inject.c	Cleanup: Switch to strlcpy from strncpy	2022-09-27 16:35:29 -07:00
zio.c	zio can deadlock during device removal	2022-12-02 17:46:29 -08:00
zle.c
zrlock.c	Micro-optimize zrl_remove()	2022-11-29 09:26:03 -08:00
zthr.c
zvol.c	zfs_rename: support RENAME_* flags	2022-10-28 09:49:20 -07:00