mirror of
https://git.proxmox.com/git/mirror_zfs.git
synced 2024-12-28 03:49:38 +03:00
1ce23dcaff
Authored by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Dan McDonald <danmcd@joyent.com>
Ported-by: Prakash Surya <prakash.surya@delphix.com>
Problem
=======
The current implementation of zil_commit() can introduce significant
latency, beyond what is inherent due to the latency of the underlying
storage. The additional latency comes from two main problems:
1. When there's outstanding ZIL blocks being written (i.e. there's
already a "writer thread" in progress), then any new calls to
zil_commit() will block waiting for the currently oustanding ZIL
blocks to complete. The blocks written for each "writer thread" is
coined a "batch", and there can only ever be a single "batch" being
written at a time. When a batch is being written, any new ZIL
transactions will have to wait for the next batch to be written,
which won't occur until the current batch finishes.
As a result, the underlying storage may not be used as efficiently
as possible. While "new" threads enter zil_commit() and are blocked
waiting for the next batch, it's possible that the underlying
storage isn't fully utilized by the current batch of ZIL blocks. In
that case, it'd be better to allow these new threads to generate
(and issue) a new ZIL block, such that it could be serviced by the
underlying storage concurrently with the other ZIL blocks that are
being serviced.
2. Any call to zil_commit() must wait for all ZIL blocks in its "batch"
to complete, prior to zil_commit() returning. The size of any given
batch is proportional to the number of ZIL transaction in the queue
at the time that the batch starts processing the queue; which
doesn't occur until the previous batch completes. Thus, if there's a
lot of transactions in the queue, the batch could be composed of
many ZIL blocks, and each call to zil_commit() will have to wait for
all of these writes to complete (even if the thread calling
zil_commit() only cared about one of the transactions in the batch).
To further complicate the situation, these two issues result in the
following side effect:
3. If a given batch takes longer to complete than normal, this results
in larger batch sizes, which then take longer to complete and
further drive up the latency of zil_commit(). This can occur for a
number of reasons, including (but not limited to): transient changes
in the workload, and storage latency irregularites.
Solution
========
The solution attempted by this change has the following goals:
1. no on-disk changes; maintain current on-disk format.
2. modify the "batch size" to be equal to the "ZIL block size".
3. allow new batches to be generated and issued to disk, while there's
already batches being serviced by the disk.
4. allow zil_commit() to wait for as few ZIL blocks as possible.
5. use as few ZIL blocks as possible, for the same amount of ZIL
transactions, without introducing significant latency to any
individual ZIL transaction. i.e. use fewer, but larger, ZIL blocks.
In theory, with these goals met, the new allgorithm will allow the
following improvements:
1. new ZIL blocks can be generated and issued, while there's already
oustanding ZIL blocks being serviced by the storage.
2. the latency of zil_commit() should be proportional to the underlying
storage latency, rather than the incoming synchronous workload.
Porting Notes
=============
Due to the changes made in commit
|
||
|---|---|---|
| .. | ||
| abd.c | ||
| arc.c | ||
| blkptr.c | ||
| bplist.c | ||
| bpobj.c | ||
| bptree.c | ||
| bqueue.c | ||
| dbuf_stats.c | ||
| dbuf.c | ||
| ddt_zap.c | ||
| ddt.c | ||
| dmu_diff.c | ||
| dmu_object.c | ||
| dmu_objset.c | ||
| dmu_send.c | ||
| dmu_traverse.c | ||
| dmu_tx.c | ||
| dmu_zfetch.c | ||
| dmu.c | ||
| dnode_sync.c | ||
| dnode.c | ||
| dsl_bookmark.c | ||
| dsl_crypt.c | ||
| dsl_dataset.c | ||
| dsl_deadlist.c | ||
| dsl_deleg.c | ||
| dsl_destroy.c | ||
| dsl_dir.c | ||
| dsl_pool.c | ||
| dsl_prop.c | ||
| dsl_scan.c | ||
| dsl_synctask.c | ||
| dsl_userhold.c | ||
| edonr_zfs.c | ||
| fm.c | ||
| gzip.c | ||
| hkdf.c | ||
| lz4.c | ||
| lzjb.c | ||
| Makefile.in | ||
| metaslab.c | ||
| mmp.c | ||
| multilist.c | ||
| pathname.c | ||
| policy.c | ||
| qat_compress.c | ||
| qat_compress.h | ||
| range_tree.c | ||
| refcount.c | ||
| rrwlock.c | ||
| sa.c | ||
| sha256.c | ||
| skein_zfs.c | ||
| spa_boot.c | ||
| spa_config.c | ||
| spa_errlog.c | ||
| spa_history.c | ||
| spa_misc.c | ||
| spa_stats.c | ||
| spa.c | ||
| space_map.c | ||
| space_reftree.c | ||
| trace.c | ||
| txg.c | ||
| uberblock.c | ||
| unique.c | ||
| vdev_cache.c | ||
| vdev_disk.c | ||
| vdev_file.c | ||
| vdev_label.c | ||
| vdev_mirror.c | ||
| vdev_missing.c | ||
| vdev_queue.c | ||
| vdev_raidz_math_aarch64_neon_common.h | ||
| vdev_raidz_math_aarch64_neon.c | ||
| vdev_raidz_math_aarch64_neonx2.c | ||
| vdev_raidz_math_avx2.c | ||
| vdev_raidz_math_avx512bw.c | ||
| vdev_raidz_math_avx512f.c | ||
| vdev_raidz_math_impl.h | ||
| vdev_raidz_math_scalar.c | ||
| vdev_raidz_math_sse2.c | ||
| vdev_raidz_math_ssse3.c | ||
| vdev_raidz_math.c | ||
| vdev_raidz.c | ||
| vdev_root.c | ||
| vdev.c | ||
| zap_leaf.c | ||
| zap_micro.c | ||
| zap.c | ||
| zfeature.c | ||
| zfs_acl.c | ||
| zfs_byteswap.c | ||
| zfs_ctldir.c | ||
| zfs_debug.c | ||
| zfs_dir.c | ||
| zfs_fm.c | ||
| zfs_fuid.c | ||
| zfs_ioctl.c | ||
| zfs_log.c | ||
| zfs_onexit.c | ||
| zfs_ratelimit.c | ||
| zfs_replay.c | ||
| zfs_rlock.c | ||
| zfs_sa.c | ||
| zfs_vfsops.c | ||
| zfs_vnops.c | ||
| zfs_znode.c | ||
| zil.c | ||
| zio_checksum.c | ||
| zio_compress.c | ||
| zio_crypt.c | ||
| zio_inject.c | ||
| zio.c | ||
| zle.c | ||
| zpl_ctldir.c | ||
| zpl_export.c | ||
| zpl_file.c | ||
| zpl_inode.c | ||
| zpl_super.c | ||
| zpl_xattr.c | ||
| zrlock.c | ||
| zvol.c | ||