Go to file
Matthew Ahrens ba67d82142
Improve zfs receive performance with lightweight write
The performance of `zfs receive` can be bottlenecked on the CPU consumed
by the `receive_writer` thread, especially when receiving streams with
small compressed block sizes.  Much of the CPU is spent creating and
destroying dbuf's and arc buf's, one for each `WRITE` record in the send
stream.

This commit introduces the concept of "lightweight writes", which allows
`zfs receive` to write to the DMU by providing an ABD, and instantiating
only a new type of `dbuf_dirty_record_t`.  The dbuf and arc buf for this
"dirty leaf block" are not instantiated.

Because there is no dbuf with the dirty data, this mechanism doesn't
support reading from "lightweight-dirty" blocks (they would see the
on-disk state rather than the dirty data).  Since the dedup-receive code
has been removed, `zfs receive` is write-only, so this works fine.

Because there are no arc bufs for the received data, the received data
is no longer cached in the ARC.

Testing a receive of a stream with average compressed block size of 4KB,
this commit improves performance by 50%, while also reducing CPU usage
by 50% of a CPU.  On a per-block basis, CPU consumed by receive_writer()
and dbuf_evict() is now 1/7th (14%) of what it was.

Baseline: 450MB/s, CPU in receive_writer() 40% + dbuf_evict() 35%
New: 670MB/s, CPU in receive_writer() 17% + dbuf_evict() 0%

The code is also restructured in a few ways:

Added a `dr_dnode` field to the dbuf_dirty_record_t.  This simplifies
some existing code that no longer needs `DB_DNODE_ENTER()` and related
routines.  The new field is needed by the lightweight-type dirty record.

To ensure that the `dr_dnode` field remains valid until the dirty record
is freed, we have to ensure that the `dnode_move()` doesn't relocate the
dnode_t.  To do this we keep a hold on the dnode until it's zio's have
completed.  This is already done by the user-accounting code
(`userquota_updates_task()`), this commit extends that so that it always
keeps the dnode hold until zio completion (see `dnode_rele_task()`).

`dn_dirty_txg` was previously zeroed when the dnode was synced.  This
was not necessary, since its meaning can be "when was this dnode last
dirtied".  This change simplifies the new `dnode_rele_task()` code.

Removed some dead code related to `DRR_WRITE_BYREF` (dedup receive).

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #11105
2020-12-11 10:26:02 -08:00
.github CI: add zloop workflow 2020-12-10 20:46:15 +03:00
cmd FreeBSD: Update usage of py-sysctl 2020-12-10 15:28:31 -08:00
config Implement memory and CPU hotplug 2020-12-10 14:09:23 -08:00
contrib dracut: use /bin/sh instead of bash as the intepreter 2020-11-28 11:02:08 -08:00
etc Verify zfs module loaded before starting services 2020-11-28 11:11:18 -08:00
include Improve zfs receive performance with lightweight write 2020-12-11 10:26:02 -08:00
lib zpool: Dryrun fails to list some devices 2020-12-04 14:04:39 -08:00
man Add -u option to 'zfs create' 2020-12-04 14:01:42 -08:00
module Improve zfs receive performance with lightweight write 2020-12-11 10:26:02 -08:00
rpm zpool_influxdb: move to libexec dir 2020-11-28 11:15:57 -08:00
scripts Add compatibility for busybox mktemp 2020-12-03 10:01:16 -08:00
tests ZTS: three small follow up fixes for #11167 2020-12-09 21:27:12 -08:00
udev Centralize variable substitution 2020-07-14 17:33:44 -07:00
.editorconfig Add an .editorconfig; document git whitespace settings 2020-01-27 13:32:52 -08:00
.gitignore Add FreeBSD support to OpenZFS 2020-04-14 11:36:28 -07:00
.gitmodules Add zimport.sh compatibility test script 2014-02-21 12:10:31 -08:00
AUTHORS Add zstd support to zfs 2020-08-20 10:30:06 -07:00
autogen.sh Cause autogen.sh to fail if autoreconf fails 2018-07-06 09:27:37 -07:00
CODE_OF_CONDUCT.md Replace ZFS on Linux references with OpenZFS 2020-10-08 20:10:13 -07:00
configure.ac Distributed Spare (dRAID) Feature 2020-11-13 13:51:51 -08:00
copy-builtin Replace ZFS on Linux references with OpenZFS 2020-10-08 20:10:13 -07:00
COPYRIGHT Fix typos 2020-06-09 21:24:09 -07:00
cppcheck-suppressions.txt Import ZStandard v1.4.5 2020-08-20 10:30:06 -07:00
LICENSE Update build system and packaging 2018-05-29 16:00:33 -07:00
Makefile.am dracut: use /bin/sh instead of bash as the intepreter 2020-11-28 11:02:08 -08:00
META Increase Supported Linux Kernel to 5.9 2020-10-13 09:51:13 -07:00
NEWS Fix NEWS file 2020-08-26 21:44:41 -07:00
NOTICE Update build system and packaging 2018-05-29 16:00:33 -07:00
README.md docs: update README's installation link 2020-10-08 09:33:53 -07:00
TEST Remove CI builder customization from TEST 2020-03-16 10:46:03 -07:00
zfs.release.in Move zfs.release generation to configure step 2012-07-12 12:22:51 -07:00

img

OpenZFS is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS community. This repository contains the code for running OpenZFS on Linux and FreeBSD.

codecov coverity

Official Resources

Installation

Full documentation for installing OpenZFS on your favorite operating system can be found at the Getting Started Page.

Contribute & Develop

We have a separate document with contribution guidelines.

We have a Code of Conduct.

Release

OpenZFS is released under a CDDL license. For more details see the NOTICE, LICENSE and COPYRIGHT files; UCRL-CODE-235197

Supported Kernels

  • The META file contains the officially recognized supported Linux kernel versions.
  • Supported FreeBSD versions are 12-STABLE and 13-CURRENT.