2008-11-20 23:01:55 +03:00
|
|
|
/*
|
|
|
|
|
* CDDL HEADER START
|
|
|
|
|
*
|
|
|
|
|
* The contents of this file are subject to the terms of the
|
|
|
|
|
* Common Development and Distribution License (the "License").
|
|
|
|
|
* You may not use this file except in compliance with the License.
|
|
|
|
|
*
|
|
|
|
|
* You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
|
2022-07-12 00:16:13 +03:00
|
|
|
* or https://opensource.org/licenses/CDDL-1.0.
|
2008-11-20 23:01:55 +03:00
|
|
|
* See the License for the specific language governing permissions
|
|
|
|
|
* and limitations under the License.
|
|
|
|
|
*
|
|
|
|
|
* When distributing Covered Code, include this CDDL HEADER in each
|
|
|
|
|
* file and include the License file at usr/src/OPENSOLARIS.LICENSE.
|
|
|
|
|
* If applicable, add the following below this CDDL HEADER, with the
|
|
|
|
|
* fields enclosed by brackets "[]" replaced with your own identifying
|
|
|
|
|
* information: Portions Copyright [yyyy] [name of copyright owner]
|
|
|
|
|
*
|
|
|
|
|
* CDDL HEADER END
|
|
|
|
|
*/
|
|
|
|
|
/*
|
2010-05-29 00:45:14 +04:00
|
|
|
* Copyright (c) 2005, 2010, Oracle and/or its affiliates. All rights reserved.
|
Improve zfs receive performance with lightweight write
The performance of `zfs receive` can be bottlenecked on the CPU consumed
by the `receive_writer` thread, especially when receiving streams with
small compressed block sizes. Much of the CPU is spent creating and
destroying dbuf's and arc buf's, one for each `WRITE` record in the send
stream.
This commit introduces the concept of "lightweight writes", which allows
`zfs receive` to write to the DMU by providing an ABD, and instantiating
only a new type of `dbuf_dirty_record_t`. The dbuf and arc buf for this
"dirty leaf block" are not instantiated.
Because there is no dbuf with the dirty data, this mechanism doesn't
support reading from "lightweight-dirty" blocks (they would see the
on-disk state rather than the dirty data). Since the dedup-receive code
has been removed, `zfs receive` is write-only, so this works fine.
Because there are no arc bufs for the received data, the received data
is no longer cached in the ARC.
Testing a receive of a stream with average compressed block size of 4KB,
this commit improves performance by 50%, while also reducing CPU usage
by 50% of a CPU. On a per-block basis, CPU consumed by receive_writer()
and dbuf_evict() is now 1/7th (14%) of what it was.
Baseline: 450MB/s, CPU in receive_writer() 40% + dbuf_evict() 35%
New: 670MB/s, CPU in receive_writer() 17% + dbuf_evict() 0%
The code is also restructured in a few ways:
Added a `dr_dnode` field to the dbuf_dirty_record_t. This simplifies
some existing code that no longer needs `DB_DNODE_ENTER()` and related
routines. The new field is needed by the lightweight-type dirty record.
To ensure that the `dr_dnode` field remains valid until the dirty record
is freed, we have to ensure that the `dnode_move()` doesn't relocate the
dnode_t. To do this we keep a hold on the dnode until it's zio's have
completed. This is already done by the user-accounting code
(`userquota_updates_task()`), this commit extends that so that it always
keeps the dnode hold until zio completion (see `dnode_rele_task()`).
`dn_dirty_txg` was previously zeroed when the dnode was synced. This
was not necessary, since its meaning can be "when was this dnode last
dirtied". This change simplifies the new `dnode_rele_task()` code.
Removed some dead code related to `DRR_WRITE_BYREF` (dedup receive).
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #11105
2020-12-11 21:26:02 +03:00
|
|
|
* Copyright (c) 2012, 2020 by Delphix. All rights reserved.
|
2015-04-02 06:44:32 +03:00
|
|
|
* Copyright (c) 2014 Spectra Logic Corporation, All rights reserved.
|
2008-11-20 23:01:55 +03:00
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#include <sys/zfs_context.h>
|
|
|
|
|
#include <sys/dbuf.h>
|
|
|
|
|
#include <sys/dnode.h>
|
|
|
|
|
#include <sys/dmu.h>
|
|
|
|
|
#include <sys/dmu_impl.h>
|
|
|
|
|
#include <sys/dmu_tx.h>
|
|
|
|
|
#include <sys/dmu_objset.h>
|
|
|
|
|
#include <sys/dsl_dir.h>
|
|
|
|
|
#include <sys/dsl_dataset.h>
|
|
|
|
|
#include <sys/spa.h>
|
|
|
|
|
#include <sys/zio.h>
|
|
|
|
|
#include <sys/dmu_zfetch.h>
|
2014-04-16 07:40:22 +04:00
|
|
|
#include <sys/range_tree.h>
|
Enable use of DTRACE_PROBE* macros in "spl" module
This change modifies some of the infrastructure for enabling the use of
the DTRACE_PROBE* macros, such that we can use tehm in the "spl" module.
Currently, when the DTRACE_PROBE* macros are used, they get expanded to
create new functions, and these dynamically generated functions become
part of the "zfs" module.
Since the "spl" module does not depend on the "zfs" module, the use of
DTRACE_PROBE* in the "spl" module would result in undefined symbols
being used in the "spl" module. Specifically, DTRACE_PROBE* would turn
into a function call, and the function being called would be a symbol
only contained in the "zfs" module; which results in a linker and/or
runtime error.
Thus, this change adds the necessary logic to the "spl" module, to
mirror the tracing functionality available to the "zfs" module. After
this change, we'll have a "trace_zfs.h" header file which defines the
probes available only to the "zfs" module, and a "trace_spl.h" header
file which defines the probes available only to the "spl" module.
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Prakash Surya <prakash.surya@delphix.com>
Closes #9525
2019-10-30 21:02:41 +03:00
|
|
|
#include <sys/trace_zfs.h>
|
2018-02-14 01:54:54 +03:00
|
|
|
#include <sys/zfs_project.h>
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
dnode_stats_t dnode_stats = {
|
|
|
|
|
{ "dnode_hold_dbuf_hold", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_dbuf_read", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_alloc_hits", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_alloc_misses", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_alloc_interior", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_alloc_lock_retry", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_alloc_lock_misses", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_alloc_type_none", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_free_hits", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_free_misses", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_free_lock_misses", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_free_lock_retry", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_free_overflow", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_hold_free_refcount", KSTAT_DATA_UINT64 },
|
2018-01-19 12:19:47 +03:00
|
|
|
{ "dnode_free_interior_lock_retry", KSTAT_DATA_UINT64 },
|
2017-09-06 02:15:04 +03:00
|
|
|
{ "dnode_allocate", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_reallocate", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_buf_evict", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_alloc_next_chunk", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_alloc_race", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_alloc_next_block", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_move_invalid", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_move_recheck1", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_move_recheck2", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_move_special", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_move_handle", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_move_rwlock", KSTAT_DATA_UINT64 },
|
|
|
|
|
{ "dnode_move_active", KSTAT_DATA_UINT64 },
|
|
|
|
|
};
|
|
|
|
|
|
2022-11-29 20:33:45 +03:00
|
|
|
dnode_sums_t dnode_sums;
|
|
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
static kstat_t *dnode_ksp;
|
2008-11-20 23:01:55 +03:00
|
|
|
static kmem_cache_t *dnode_cache;
|
|
|
|
|
|
2019-12-05 23:37:00 +03:00
|
|
|
static dnode_phys_t dnode_phys_zero __maybe_unused;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
int zfs_default_bs = SPA_MINBLOCKSHIFT;
|
|
|
|
|
int zfs_default_ibs = DN_MAX_INDBLKSHIFT;
|
|
|
|
|
|
2010-08-27 21:55:07 +04:00
|
|
|
#ifdef _KERNEL
|
2010-08-27 01:24:34 +04:00
|
|
|
static kmem_cbrc_t dnode_move(void *, void *, size_t, void *);
|
2010-08-27 21:55:07 +04:00
|
|
|
#endif /* _KERNEL */
|
2010-08-27 01:24:34 +04:00
|
|
|
|
2015-04-03 06:14:28 +03:00
|
|
|
static int
|
|
|
|
|
dbuf_compare(const void *x1, const void *x2)
|
|
|
|
|
{
|
|
|
|
|
const dmu_buf_impl_t *d1 = x1;
|
|
|
|
|
const dmu_buf_impl_t *d2 = x2;
|
|
|
|
|
|
Reduce loaded range tree memory usage
This patch implements a new tree structure for ZFS, and uses it to
store range trees more efficiently.
The new structure is approximately a B-tree, though there are some
small differences from the usual characterizations. The tree has core
nodes and leaf nodes; each contain data elements, which the elements
in the core nodes acting as separators between its children. The
difference between core and leaf nodes is that the core nodes have an
array of children, while leaf nodes don't. Every node in the tree may
be only partially full; in most cases, they are all at least 50% full
(in terms of element count) except for the root node, which can be
less full. Underfull nodes will steal from their neighbors or merge to
remain full enough, while overfull nodes will split in two. The data
elements are contained in tree-controlled buffers; they are copied
into these on insertion, and overwritten on deletion. This means that
the elements are not independently allocated, which reduces overhead,
but also means they can't be shared between trees (and also that
pointers to them are only valid until a side-effectful tree operation
occurs). The overhead varies based on how dense the tree is, but is
usually on the order of about 50% of the element size; the per-node
overheads are very small, and so don't make a significant difference.
The trees can accept arbitrary records; they accept a size and a
comparator to allow them to be used for a variety of purposes.
The new trees replace the AVL trees used in the range trees today.
Currently, the range_seg_t structure contains three 8 byte integers
of payload and two 24 byte avl_tree_node_ts to handle its storage in
both an offset-sorted tree and a size-sorted tree (total size: 64
bytes). In the new model, the range seg structures are usually two 4
byte integers, but a separate one needs to exist for the size-sorted
and offset-sorted tree. Between the raw size, the 50% overhead, and
the double storage, the new btrees are expected to use 8*1.5*2 = 24
bytes per record, or 33.3% as much memory as the AVL trees (this is
for the purposes of storing metaslab range trees; for other purposes,
like scrubs, they use ~50% as much memory).
We reduced the size of the payload in the range segments by teaching
range trees about starting offsets and shifts; since metaslabs have a
fixed starting offset, and they all operate in terms of disk sectors,
we can store the ranges using 4-byte integers as long as the size of
the metaslab divided by the sector size is less than 2^32. For 512-byte
sectors, this is a 2^41 (or 2TB) metaslab, which with the default
settings corresponds to a 256PB disk. 4k sector disks can handle
metaslabs up to 2^46 bytes, or 2^63 byte disks. Since we do not
anticipate disks of this size in the near future, there should be
almost no cases where metaslabs need 64-byte integers to store their
ranges. We do still have the capability to store 64-byte integer ranges
to account for cases where we are storing per-vdev (or per-dnode) trees,
which could reasonably go above the limits discussed. We also do not
store fill information in the compact version of the node, since it
is only used for sorted scrub.
We also optimized the metaslab loading process in various other ways
to offset some inefficiencies in the btree model. While individual
operations (find, insert, remove_from) are faster for the btree than
they are for the avl tree, remove usually requires a find operation,
while in the AVL tree model the element itself suffices. Some clever
changes actually caused an overall speedup in metaslab loading; we use
approximately 40% less cpu to load metaslabs in our tests on Illumos.
Another memory and performance optimization was achieved by changing
what is stored in the size-sorted trees. When a disk is heavily
fragmented, the df algorithm used by default in ZFS will almost always
find a number of small regions in its initial cursor-based search; it
will usually only fall back to the size-sorted tree to find larger
regions. If we increase the size of the cursor-based search slightly,
and don't store segments that are smaller than a tunable size floor
in the size-sorted tree, we can further cut memory usage down to
below 20% of what the AVL trees store. This also results in further
reductions in CPU time spent loading metaslabs.
The 16KiB size floor was chosen because it results in substantial memory
usage reduction while not usually resulting in situations where we can't
find an appropriate chunk with the cursor and are forced to use an
oversized chunk from the size-sorted tree. In addition, even if we do
have to use an oversized chunk from the size-sorted tree, the chunk
would be too small to use for ZIL allocations, so it isn't as big of a
loss as it might otherwise be. And often, more small allocations will
follow the initial one, and the cursor search will now find the
remainder of the chunk we didn't use all of and use it for subsequent
allocations. Practical testing has shown little or no change in
fragmentation as a result of this change.
If the size-sorted tree becomes empty while the offset sorted one still
has entries, it will load all the entries from the offset sorted tree
and disregard the size floor until it is unloaded again. This operation
occurs rarely with the default setting, only on incredibly thoroughly
fragmented pools.
There are some other small changes to zdb to teach it to handle btrees,
but nothing major.
Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Matt Ahrens <matt@delphix.com>
Reviewed by: Sebastien Roy seb@delphix.com
Reviewed-by: Igor Kozhukhov <igor@dilos.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #9181
2019-10-09 20:36:03 +03:00
|
|
|
int cmp = TREE_CMP(d1->db_level, d2->db_level);
|
2016-08-27 21:12:53 +03:00
|
|
|
if (likely(cmp))
|
|
|
|
|
return (cmp);
|
2015-04-03 06:14:28 +03:00
|
|
|
|
Reduce loaded range tree memory usage
This patch implements a new tree structure for ZFS, and uses it to
store range trees more efficiently.
The new structure is approximately a B-tree, though there are some
small differences from the usual characterizations. The tree has core
nodes and leaf nodes; each contain data elements, which the elements
in the core nodes acting as separators between its children. The
difference between core and leaf nodes is that the core nodes have an
array of children, while leaf nodes don't. Every node in the tree may
be only partially full; in most cases, they are all at least 50% full
(in terms of element count) except for the root node, which can be
less full. Underfull nodes will steal from their neighbors or merge to
remain full enough, while overfull nodes will split in two. The data
elements are contained in tree-controlled buffers; they are copied
into these on insertion, and overwritten on deletion. This means that
the elements are not independently allocated, which reduces overhead,
but also means they can't be shared between trees (and also that
pointers to them are only valid until a side-effectful tree operation
occurs). The overhead varies based on how dense the tree is, but is
usually on the order of about 50% of the element size; the per-node
overheads are very small, and so don't make a significant difference.
The trees can accept arbitrary records; they accept a size and a
comparator to allow them to be used for a variety of purposes.
The new trees replace the AVL trees used in the range trees today.
Currently, the range_seg_t structure contains three 8 byte integers
of payload and two 24 byte avl_tree_node_ts to handle its storage in
both an offset-sorted tree and a size-sorted tree (total size: 64
bytes). In the new model, the range seg structures are usually two 4
byte integers, but a separate one needs to exist for the size-sorted
and offset-sorted tree. Between the raw size, the 50% overhead, and
the double storage, the new btrees are expected to use 8*1.5*2 = 24
bytes per record, or 33.3% as much memory as the AVL trees (this is
for the purposes of storing metaslab range trees; for other purposes,
like scrubs, they use ~50% as much memory).
We reduced the size of the payload in the range segments by teaching
range trees about starting offsets and shifts; since metaslabs have a
fixed starting offset, and they all operate in terms of disk sectors,
we can store the ranges using 4-byte integers as long as the size of
the metaslab divided by the sector size is less than 2^32. For 512-byte
sectors, this is a 2^41 (or 2TB) metaslab, which with the default
settings corresponds to a 256PB disk. 4k sector disks can handle
metaslabs up to 2^46 bytes, or 2^63 byte disks. Since we do not
anticipate disks of this size in the near future, there should be
almost no cases where metaslabs need 64-byte integers to store their
ranges. We do still have the capability to store 64-byte integer ranges
to account for cases where we are storing per-vdev (or per-dnode) trees,
which could reasonably go above the limits discussed. We also do not
store fill information in the compact version of the node, since it
is only used for sorted scrub.
We also optimized the metaslab loading process in various other ways
to offset some inefficiencies in the btree model. While individual
operations (find, insert, remove_from) are faster for the btree than
they are for the avl tree, remove usually requires a find operation,
while in the AVL tree model the element itself suffices. Some clever
changes actually caused an overall speedup in metaslab loading; we use
approximately 40% less cpu to load metaslabs in our tests on Illumos.
Another memory and performance optimization was achieved by changing
what is stored in the size-sorted trees. When a disk is heavily
fragmented, the df algorithm used by default in ZFS will almost always
find a number of small regions in its initial cursor-based search; it
will usually only fall back to the size-sorted tree to find larger
regions. If we increase the size of the cursor-based search slightly,
and don't store segments that are smaller than a tunable size floor
in the size-sorted tree, we can further cut memory usage down to
below 20% of what the AVL trees store. This also results in further
reductions in CPU time spent loading metaslabs.
The 16KiB size floor was chosen because it results in substantial memory
usage reduction while not usually resulting in situations where we can't
find an appropriate chunk with the cursor and are forced to use an
oversized chunk from the size-sorted tree. In addition, even if we do
have to use an oversized chunk from the size-sorted tree, the chunk
would be too small to use for ZIL allocations, so it isn't as big of a
loss as it might otherwise be. And often, more small allocations will
follow the initial one, and the cursor search will now find the
remainder of the chunk we didn't use all of and use it for subsequent
allocations. Practical testing has shown little or no change in
fragmentation as a result of this change.
If the size-sorted tree becomes empty while the offset sorted one still
has entries, it will load all the entries from the offset sorted tree
and disregard the size floor until it is unloaded again. This operation
occurs rarely with the default setting, only on incredibly thoroughly
fragmented pools.
There are some other small changes to zdb to teach it to handle btrees,
but nothing major.
Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Matt Ahrens <matt@delphix.com>
Reviewed by: Sebastien Roy seb@delphix.com
Reviewed-by: Igor Kozhukhov <igor@dilos.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #9181
2019-10-09 20:36:03 +03:00
|
|
|
cmp = TREE_CMP(d1->db_blkid, d2->db_blkid);
|
2016-08-27 21:12:53 +03:00
|
|
|
if (likely(cmp))
|
|
|
|
|
return (cmp);
|
2015-04-03 06:14:28 +03:00
|
|
|
|
2015-05-06 20:08:25 +03:00
|
|
|
if (d1->db_state == DB_SEARCH) {
|
|
|
|
|
ASSERT3S(d2->db_state, !=, DB_SEARCH);
|
2015-04-03 06:14:28 +03:00
|
|
|
return (-1);
|
2015-05-06 20:08:25 +03:00
|
|
|
} else if (d2->db_state == DB_SEARCH) {
|
|
|
|
|
ASSERT3S(d1->db_state, !=, DB_SEARCH);
|
2015-04-01 18:10:58 +03:00
|
|
|
return (1);
|
|
|
|
|
}
|
|
|
|
|
|
Reduce loaded range tree memory usage
This patch implements a new tree structure for ZFS, and uses it to
store range trees more efficiently.
The new structure is approximately a B-tree, though there are some
small differences from the usual characterizations. The tree has core
nodes and leaf nodes; each contain data elements, which the elements
in the core nodes acting as separators between its children. The
difference between core and leaf nodes is that the core nodes have an
array of children, while leaf nodes don't. Every node in the tree may
be only partially full; in most cases, they are all at least 50% full
(in terms of element count) except for the root node, which can be
less full. Underfull nodes will steal from their neighbors or merge to
remain full enough, while overfull nodes will split in two. The data
elements are contained in tree-controlled buffers; they are copied
into these on insertion, and overwritten on deletion. This means that
the elements are not independently allocated, which reduces overhead,
but also means they can't be shared between trees (and also that
pointers to them are only valid until a side-effectful tree operation
occurs). The overhead varies based on how dense the tree is, but is
usually on the order of about 50% of the element size; the per-node
overheads are very small, and so don't make a significant difference.
The trees can accept arbitrary records; they accept a size and a
comparator to allow them to be used for a variety of purposes.
The new trees replace the AVL trees used in the range trees today.
Currently, the range_seg_t structure contains three 8 byte integers
of payload and two 24 byte avl_tree_node_ts to handle its storage in
both an offset-sorted tree and a size-sorted tree (total size: 64
bytes). In the new model, the range seg structures are usually two 4
byte integers, but a separate one needs to exist for the size-sorted
and offset-sorted tree. Between the raw size, the 50% overhead, and
the double storage, the new btrees are expected to use 8*1.5*2 = 24
bytes per record, or 33.3% as much memory as the AVL trees (this is
for the purposes of storing metaslab range trees; for other purposes,
like scrubs, they use ~50% as much memory).
We reduced the size of the payload in the range segments by teaching
range trees about starting offsets and shifts; since metaslabs have a
fixed starting offset, and they all operate in terms of disk sectors,
we can store the ranges using 4-byte integers as long as the size of
the metaslab divided by the sector size is less than 2^32. For 512-byte
sectors, this is a 2^41 (or 2TB) metaslab, which with the default
settings corresponds to a 256PB disk. 4k sector disks can handle
metaslabs up to 2^46 bytes, or 2^63 byte disks. Since we do not
anticipate disks of this size in the near future, there should be
almost no cases where metaslabs need 64-byte integers to store their
ranges. We do still have the capability to store 64-byte integer ranges
to account for cases where we are storing per-vdev (or per-dnode) trees,
which could reasonably go above the limits discussed. We also do not
store fill information in the compact version of the node, since it
is only used for sorted scrub.
We also optimized the metaslab loading process in various other ways
to offset some inefficiencies in the btree model. While individual
operations (find, insert, remove_from) are faster for the btree than
they are for the avl tree, remove usually requires a find operation,
while in the AVL tree model the element itself suffices. Some clever
changes actually caused an overall speedup in metaslab loading; we use
approximately 40% less cpu to load metaslabs in our tests on Illumos.
Another memory and performance optimization was achieved by changing
what is stored in the size-sorted trees. When a disk is heavily
fragmented, the df algorithm used by default in ZFS will almost always
find a number of small regions in its initial cursor-based search; it
will usually only fall back to the size-sorted tree to find larger
regions. If we increase the size of the cursor-based search slightly,
and don't store segments that are smaller than a tunable size floor
in the size-sorted tree, we can further cut memory usage down to
below 20% of what the AVL trees store. This also results in further
reductions in CPU time spent loading metaslabs.
The 16KiB size floor was chosen because it results in substantial memory
usage reduction while not usually resulting in situations where we can't
find an appropriate chunk with the cursor and are forced to use an
oversized chunk from the size-sorted tree. In addition, even if we do
have to use an oversized chunk from the size-sorted tree, the chunk
would be too small to use for ZIL allocations, so it isn't as big of a
loss as it might otherwise be. And often, more small allocations will
follow the initial one, and the cursor search will now find the
remainder of the chunk we didn't use all of and use it for subsequent
allocations. Practical testing has shown little or no change in
fragmentation as a result of this change.
If the size-sorted tree becomes empty while the offset sorted one still
has entries, it will load all the entries from the offset sorted tree
and disregard the size floor until it is unloaded again. This operation
occurs rarely with the default setting, only on incredibly thoroughly
fragmented pools.
There are some other small changes to zdb to teach it to handle btrees,
but nothing major.
Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Matt Ahrens <matt@delphix.com>
Reviewed by: Sebastien Roy seb@delphix.com
Reviewed-by: Igor Kozhukhov <igor@dilos.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #9181
2019-10-09 20:36:03 +03:00
|
|
|
return (TREE_PCMP(d1, d2));
|
2015-04-03 06:14:28 +03:00
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
static int
|
|
|
|
|
dnode_cons(void *arg, void *unused, int kmflag)
|
|
|
|
|
{
|
2021-12-12 18:06:44 +03:00
|
|
|
(void) unused, (void) kmflag;
|
2008-11-20 23:01:55 +03:00
|
|
|
dnode_t *dn = arg;
|
|
|
|
|
|
Identify locks flagged by lockdep
When running a kernel with CONFIG_LOCKDEP=y, lockdep reports possible
recursive locking in some cases and possible circular locking dependency
in others, within the SPL and ZFS modules.
This patch uses a mutex type defined in SPL, MUTEX_NOLOCKDEP, to mark
such mutexes when they are initialized. This mutex type causes
attempts to take or release those locks to be wrapped in lockdep_off()
and lockdep_on() calls to silence the dependency checker and allow the
use of lock_stats to examine contention.
For RW locks, it uses an analogous lock type, RW_NOLOCKDEP.
The goal is that these locks are ultimately changed back to type
MUTEX_DEFAULT or RW_DEFAULT, after the locks are annotated to reflect
their relationship (e.g. z_name_lock below) or any real problem with the
lock dependencies are fixed.
Some of the affected locks are:
tc_open_lock:
=============
This is an array of locks, all with same name, which txg_quiesce must
take all of in order to move txg to next state. All default to the same
lockdep class, and so to lockdep appears recursive.
zp->z_name_lock:
================
In zfs_rmdir,
dzp = znode for the directory (input to zfs_dirent_lock)
zp = znode for the entry being removed (output of zfs_dirent_lock)
zfs_rmdir()->zfs_dirent_lock() takes z_name_lock in dzp
zfs_rmdir() takes z_name_lock in zp
Since both dzp and zp are type znode_t, the locks have the same default
class, and lockdep considers it a possible recursive lock attempt.
l->l_rwlock:
============
zap_expand_leaf() sometimes creates two new zap leaf structures, via
these call paths:
zap_deref_leaf()->zap_get_leaf_byblk()->zap_leaf_open()
zap_expand_leaf()->zap_create_leaf()->zap_expand_leaf()->zap_create_leaf()
Because both zap_leaf_open() and zap_create_leaf() initialize
l->l_rwlock in their (separate) leaf structures, the lockdep class is
the same, and the linux kernel believes these might both be the same
lock, and emits a possible recursive lock warning.
Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3895
2015-10-15 23:08:27 +03:00
|
|
|
rw_init(&dn->dn_struct_rwlock, NULL, RW_NOLOCKDEP, NULL);
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_init(&dn->dn_mtx, NULL, MUTEX_DEFAULT, NULL);
|
|
|
|
|
mutex_init(&dn->dn_dbufs_mtx, NULL, MUTEX_DEFAULT, NULL);
|
2009-01-16 00:59:39 +03:00
|
|
|
cv_init(&dn->dn_notxholds, NULL, CV_DEFAULT, NULL);
|
2020-03-12 20:25:56 +03:00
|
|
|
cv_init(&dn->dn_nodnholds, NULL, CV_DEFAULT, NULL);
|
2009-01-16 00:59:39 +03:00
|
|
|
|
2013-09-04 16:00:57 +04:00
|
|
|
/*
|
|
|
|
|
* Every dbuf has a reference, and dropping a tracked reference is
|
|
|
|
|
* O(number of references), so don't track dn_holds.
|
|
|
|
|
*/
|
2018-10-01 20:42:05 +03:00
|
|
|
zfs_refcount_create_untracked(&dn->dn_holds);
|
|
|
|
|
zfs_refcount_create(&dn->dn_tx_holds);
|
2010-08-27 01:24:34 +04:00
|
|
|
list_link_init(&dn->dn_link);
|
|
|
|
|
|
2022-02-25 16:26:54 +03:00
|
|
|
memset(dn->dn_next_type, 0, sizeof (dn->dn_next_type));
|
|
|
|
|
memset(dn->dn_next_nblkptr, 0, sizeof (dn->dn_next_nblkptr));
|
|
|
|
|
memset(dn->dn_next_nlevels, 0, sizeof (dn->dn_next_nlevels));
|
|
|
|
|
memset(dn->dn_next_indblkshift, 0, sizeof (dn->dn_next_indblkshift));
|
|
|
|
|
memset(dn->dn_next_bonustype, 0, sizeof (dn->dn_next_bonustype));
|
|
|
|
|
memset(dn->dn_rm_spillblk, 0, sizeof (dn->dn_rm_spillblk));
|
|
|
|
|
memset(dn->dn_next_bonuslen, 0, sizeof (dn->dn_next_bonuslen));
|
|
|
|
|
memset(dn->dn_next_blksz, 0, sizeof (dn->dn_next_blksz));
|
|
|
|
|
memset(dn->dn_next_maxblkid, 0, sizeof (dn->dn_next_maxblkid));
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2021-12-12 18:06:44 +03:00
|
|
|
for (int i = 0; i < TXG_SIZE; i++) {
|
2018-04-10 21:15:05 +03:00
|
|
|
multilist_link_init(&dn->dn_dirty_link[i]);
|
2014-04-16 07:40:22 +04:00
|
|
|
dn->dn_free_ranges[i] = NULL;
|
2008-11-20 23:01:55 +03:00
|
|
|
list_create(&dn->dn_dirty_records[i],
|
|
|
|
|
sizeof (dbuf_dirty_record_t),
|
|
|
|
|
offsetof(dbuf_dirty_record_t, dr_dirty_node));
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
dn->dn_allocated_txg = 0;
|
|
|
|
|
dn->dn_free_txg = 0;
|
|
|
|
|
dn->dn_assigned_txg = 0;
|
2018-04-10 21:15:05 +03:00
|
|
|
dn->dn_dirty_txg = 0;
|
2010-08-27 01:24:34 +04:00
|
|
|
dn->dn_dirtyctx = 0;
|
|
|
|
|
dn->dn_dirtyctx_firstset = NULL;
|
|
|
|
|
dn->dn_bonus = NULL;
|
|
|
|
|
dn->dn_have_spill = B_FALSE;
|
|
|
|
|
dn->dn_zio = NULL;
|
|
|
|
|
dn->dn_oldused = 0;
|
|
|
|
|
dn->dn_oldflags = 0;
|
|
|
|
|
dn->dn_olduid = 0;
|
|
|
|
|
dn->dn_oldgid = 0;
|
2018-02-14 01:54:54 +03:00
|
|
|
dn->dn_oldprojid = ZFS_DEFAULT_PROJID;
|
2010-08-27 01:24:34 +04:00
|
|
|
dn->dn_newuid = 0;
|
|
|
|
|
dn->dn_newgid = 0;
|
2018-02-14 01:54:54 +03:00
|
|
|
dn->dn_newprojid = ZFS_DEFAULT_PROJID;
|
2010-08-27 01:24:34 +04:00
|
|
|
dn->dn_id_flags = 0;
|
|
|
|
|
|
|
|
|
|
dn->dn_dbufs_count = 0;
|
2015-04-03 06:14:28 +03:00
|
|
|
avl_create(&dn->dn_dbufs, dbuf_compare, sizeof (dmu_buf_impl_t),
|
2008-11-20 23:01:55 +03:00
|
|
|
offsetof(dmu_buf_impl_t, db_link));
|
|
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
dn->dn_moved = 0;
|
2008-11-20 23:01:55 +03:00
|
|
|
return (0);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void
|
|
|
|
|
dnode_dest(void *arg, void *unused)
|
|
|
|
|
{
|
2021-12-12 18:06:44 +03:00
|
|
|
(void) unused;
|
2008-11-20 23:01:55 +03:00
|
|
|
dnode_t *dn = arg;
|
|
|
|
|
|
|
|
|
|
rw_destroy(&dn->dn_struct_rwlock);
|
|
|
|
|
mutex_destroy(&dn->dn_mtx);
|
|
|
|
|
mutex_destroy(&dn->dn_dbufs_mtx);
|
2009-01-16 00:59:39 +03:00
|
|
|
cv_destroy(&dn->dn_notxholds);
|
2020-03-12 20:25:56 +03:00
|
|
|
cv_destroy(&dn->dn_nodnholds);
|
2018-10-01 20:42:05 +03:00
|
|
|
zfs_refcount_destroy(&dn->dn_holds);
|
|
|
|
|
zfs_refcount_destroy(&dn->dn_tx_holds);
|
2010-08-27 01:24:34 +04:00
|
|
|
ASSERT(!list_link_active(&dn->dn_link));
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2021-12-12 18:06:44 +03:00
|
|
|
for (int i = 0; i < TXG_SIZE; i++) {
|
2018-04-10 21:15:05 +03:00
|
|
|
ASSERT(!multilist_link_active(&dn->dn_dirty_link[i]));
|
2014-04-16 07:40:22 +04:00
|
|
|
ASSERT3P(dn->dn_free_ranges[i], ==, NULL);
|
2008-11-20 23:01:55 +03:00
|
|
|
list_destroy(&dn->dn_dirty_records[i]);
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(dn->dn_next_nblkptr[i]);
|
|
|
|
|
ASSERT0(dn->dn_next_nlevels[i]);
|
|
|
|
|
ASSERT0(dn->dn_next_indblkshift[i]);
|
|
|
|
|
ASSERT0(dn->dn_next_bonustype[i]);
|
|
|
|
|
ASSERT0(dn->dn_rm_spillblk[i]);
|
|
|
|
|
ASSERT0(dn->dn_next_bonuslen[i]);
|
|
|
|
|
ASSERT0(dn->dn_next_blksz[i]);
|
2017-11-08 22:12:59 +03:00
|
|
|
ASSERT0(dn->dn_next_maxblkid[i]);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(dn->dn_allocated_txg);
|
|
|
|
|
ASSERT0(dn->dn_free_txg);
|
|
|
|
|
ASSERT0(dn->dn_assigned_txg);
|
2018-04-10 21:15:05 +03:00
|
|
|
ASSERT0(dn->dn_dirty_txg);
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(dn->dn_dirtyctx);
|
2010-08-27 01:24:34 +04:00
|
|
|
ASSERT3P(dn->dn_dirtyctx_firstset, ==, NULL);
|
|
|
|
|
ASSERT3P(dn->dn_bonus, ==, NULL);
|
|
|
|
|
ASSERT(!dn->dn_have_spill);
|
|
|
|
|
ASSERT3P(dn->dn_zio, ==, NULL);
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(dn->dn_oldused);
|
|
|
|
|
ASSERT0(dn->dn_oldflags);
|
|
|
|
|
ASSERT0(dn->dn_olduid);
|
|
|
|
|
ASSERT0(dn->dn_oldgid);
|
2018-02-14 01:54:54 +03:00
|
|
|
ASSERT0(dn->dn_oldprojid);
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(dn->dn_newuid);
|
|
|
|
|
ASSERT0(dn->dn_newgid);
|
2018-02-14 01:54:54 +03:00
|
|
|
ASSERT0(dn->dn_newprojid);
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(dn->dn_id_flags);
|
|
|
|
|
|
|
|
|
|
ASSERT0(dn->dn_dbufs_count);
|
2015-04-03 06:14:28 +03:00
|
|
|
avl_destroy(&dn->dn_dbufs);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
2022-11-29 20:33:45 +03:00
|
|
|
static int
|
|
|
|
|
dnode_kstats_update(kstat_t *ksp, int rw)
|
|
|
|
|
{
|
|
|
|
|
dnode_stats_t *ds = ksp->ks_data;
|
|
|
|
|
|
|
|
|
|
if (rw == KSTAT_WRITE)
|
|
|
|
|
return (EACCES);
|
|
|
|
|
ds->dnode_hold_dbuf_hold.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_dbuf_hold);
|
|
|
|
|
ds->dnode_hold_dbuf_read.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_dbuf_read);
|
|
|
|
|
ds->dnode_hold_alloc_hits.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_alloc_hits);
|
|
|
|
|
ds->dnode_hold_alloc_misses.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_alloc_misses);
|
|
|
|
|
ds->dnode_hold_alloc_interior.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_alloc_interior);
|
|
|
|
|
ds->dnode_hold_alloc_lock_retry.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_alloc_lock_retry);
|
|
|
|
|
ds->dnode_hold_alloc_lock_misses.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_alloc_lock_misses);
|
|
|
|
|
ds->dnode_hold_alloc_type_none.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_alloc_type_none);
|
|
|
|
|
ds->dnode_hold_free_hits.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_free_hits);
|
|
|
|
|
ds->dnode_hold_free_misses.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_free_misses);
|
|
|
|
|
ds->dnode_hold_free_lock_misses.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_free_lock_misses);
|
|
|
|
|
ds->dnode_hold_free_lock_retry.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_free_lock_retry);
|
|
|
|
|
ds->dnode_hold_free_refcount.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_free_refcount);
|
|
|
|
|
ds->dnode_hold_free_overflow.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_hold_free_overflow);
|
|
|
|
|
ds->dnode_free_interior_lock_retry.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_free_interior_lock_retry);
|
|
|
|
|
ds->dnode_allocate.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_allocate);
|
|
|
|
|
ds->dnode_reallocate.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_reallocate);
|
|
|
|
|
ds->dnode_buf_evict.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_buf_evict);
|
|
|
|
|
ds->dnode_alloc_next_chunk.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_alloc_next_chunk);
|
|
|
|
|
ds->dnode_alloc_race.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_alloc_race);
|
|
|
|
|
ds->dnode_alloc_next_block.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_alloc_next_block);
|
|
|
|
|
ds->dnode_move_invalid.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_move_invalid);
|
|
|
|
|
ds->dnode_move_recheck1.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_move_recheck1);
|
|
|
|
|
ds->dnode_move_recheck2.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_move_recheck2);
|
|
|
|
|
ds->dnode_move_special.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_move_special);
|
|
|
|
|
ds->dnode_move_handle.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_move_handle);
|
|
|
|
|
ds->dnode_move_rwlock.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_move_rwlock);
|
|
|
|
|
ds->dnode_move_active.value.ui64 =
|
|
|
|
|
wmsum_value(&dnode_sums.dnode_move_active);
|
|
|
|
|
return (0);
|
|
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
void
|
|
|
|
|
dnode_init(void)
|
|
|
|
|
{
|
2010-08-27 01:24:34 +04:00
|
|
|
ASSERT(dnode_cache == NULL);
|
2011-11-02 03:56:48 +04:00
|
|
|
dnode_cache = kmem_cache_create("dnode_t", sizeof (dnode_t),
|
2014-05-15 05:17:39 +04:00
|
|
|
0, dnode_cons, dnode_dest, NULL, NULL, NULL, 0);
|
2010-08-27 01:24:34 +04:00
|
|
|
kmem_cache_set_move(dnode_cache, dnode_move);
|
2017-09-06 02:15:04 +03:00
|
|
|
|
2022-11-29 20:33:45 +03:00
|
|
|
wmsum_init(&dnode_sums.dnode_hold_dbuf_hold, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_dbuf_read, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_alloc_hits, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_alloc_misses, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_alloc_interior, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_alloc_lock_retry, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_alloc_lock_misses, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_alloc_type_none, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_free_hits, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_free_misses, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_free_lock_misses, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_free_lock_retry, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_free_refcount, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_hold_free_overflow, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_free_interior_lock_retry, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_allocate, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_reallocate, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_buf_evict, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_alloc_next_chunk, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_alloc_race, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_alloc_next_block, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_move_invalid, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_move_recheck1, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_move_recheck2, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_move_special, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_move_handle, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_move_rwlock, 0);
|
|
|
|
|
wmsum_init(&dnode_sums.dnode_move_active, 0);
|
|
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
dnode_ksp = kstat_create("zfs", 0, "dnodestats", "misc",
|
|
|
|
|
KSTAT_TYPE_NAMED, sizeof (dnode_stats) / sizeof (kstat_named_t),
|
|
|
|
|
KSTAT_FLAG_VIRTUAL);
|
|
|
|
|
if (dnode_ksp != NULL) {
|
|
|
|
|
dnode_ksp->ks_data = &dnode_stats;
|
2022-11-29 20:33:45 +03:00
|
|
|
dnode_ksp->ks_update = dnode_kstats_update;
|
2017-09-06 02:15:04 +03:00
|
|
|
kstat_install(dnode_ksp);
|
|
|
|
|
}
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
dnode_fini(void)
|
|
|
|
|
{
|
2017-09-06 02:15:04 +03:00
|
|
|
if (dnode_ksp != NULL) {
|
|
|
|
|
kstat_delete(dnode_ksp);
|
|
|
|
|
dnode_ksp = NULL;
|
|
|
|
|
}
|
|
|
|
|
|
2022-11-29 20:33:45 +03:00
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_dbuf_hold);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_dbuf_read);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_alloc_hits);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_alloc_misses);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_alloc_interior);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_alloc_lock_retry);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_alloc_lock_misses);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_alloc_type_none);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_free_hits);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_free_misses);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_free_lock_misses);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_free_lock_retry);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_free_refcount);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_hold_free_overflow);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_free_interior_lock_retry);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_allocate);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_reallocate);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_buf_evict);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_alloc_next_chunk);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_alloc_race);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_alloc_next_block);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_move_invalid);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_move_recheck1);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_move_recheck2);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_move_special);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_move_handle);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_move_rwlock);
|
|
|
|
|
wmsum_fini(&dnode_sums.dnode_move_active);
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
kmem_cache_destroy(dnode_cache);
|
2010-08-27 01:24:34 +04:00
|
|
|
dnode_cache = NULL;
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#ifdef ZFS_DEBUG
|
|
|
|
|
void
|
|
|
|
|
dnode_verify(dnode_t *dn)
|
|
|
|
|
{
|
|
|
|
|
int drop_struct_lock = FALSE;
|
|
|
|
|
|
|
|
|
|
ASSERT(dn->dn_phys);
|
|
|
|
|
ASSERT(dn->dn_objset);
|
2010-08-27 01:24:34 +04:00
|
|
|
ASSERT(dn->dn_handle->dnh_dnode == dn);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2012-12-14 03:24:15 +04:00
|
|
|
ASSERT(DMU_OT_IS_VALID(dn->dn_phys->dn_type));
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
if (!(zfs_flags & ZFS_DEBUG_DNODE_VERIFY))
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
if (!RW_WRITE_HELD(&dn->dn_struct_rwlock)) {
|
|
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_READER);
|
|
|
|
|
drop_struct_lock = TRUE;
|
|
|
|
|
}
|
|
|
|
|
if (dn->dn_phys->dn_type != DMU_OT_NONE || dn->dn_allocated_txg != 0) {
|
|
|
|
|
int i;
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
int max_bonuslen = DN_SLOTS_TO_BONUSLEN(dn->dn_num_slots);
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT3U(dn->dn_indblkshift, <=, SPA_MAXBLOCKSHIFT);
|
|
|
|
|
if (dn->dn_datablkshift) {
|
|
|
|
|
ASSERT3U(dn->dn_datablkshift, >=, SPA_MINBLOCKSHIFT);
|
|
|
|
|
ASSERT3U(dn->dn_datablkshift, <=, SPA_MAXBLOCKSHIFT);
|
|
|
|
|
ASSERT3U(1<<dn->dn_datablkshift, ==, dn->dn_datablksz);
|
|
|
|
|
}
|
|
|
|
|
ASSERT3U(dn->dn_nlevels, <=, 30);
|
2012-12-14 03:24:15 +04:00
|
|
|
ASSERT(DMU_OT_IS_VALID(dn->dn_type));
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT3U(dn->dn_nblkptr, >=, 1);
|
|
|
|
|
ASSERT3U(dn->dn_nblkptr, <=, DN_MAX_NBLKPTR);
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
ASSERT3U(dn->dn_bonuslen, <=, max_bonuslen);
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT3U(dn->dn_datablksz, ==,
|
|
|
|
|
dn->dn_datablkszsec << SPA_MINBLOCKSHIFT);
|
|
|
|
|
ASSERT3U(ISP2(dn->dn_datablksz), ==, dn->dn_datablkshift != 0);
|
|
|
|
|
ASSERT3U((dn->dn_nblkptr - 1) * sizeof (blkptr_t) +
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
dn->dn_bonuslen, <=, max_bonuslen);
|
2008-11-20 23:01:55 +03:00
|
|
|
for (i = 0; i < TXG_SIZE; i++) {
|
|
|
|
|
ASSERT3U(dn->dn_next_nlevels[i], <=, dn->dn_nlevels);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
if (dn->dn_phys->dn_type != DMU_OT_NONE)
|
|
|
|
|
ASSERT3U(dn->dn_phys->dn_nlevels, <=, dn->dn_nlevels);
|
2009-07-03 02:44:48 +04:00
|
|
|
ASSERT(DMU_OBJECT_IS_SPECIAL(dn->dn_object) || dn->dn_dbuf != NULL);
|
2008-11-20 23:01:55 +03:00
|
|
|
if (dn->dn_dbuf != NULL) {
|
|
|
|
|
ASSERT3P(dn->dn_phys, ==,
|
|
|
|
|
(dnode_phys_t *)dn->dn_dbuf->db.db_data +
|
|
|
|
|
(dn->dn_object % (dn->dn_dbuf->db.db_size >> DNODE_SHIFT)));
|
|
|
|
|
}
|
|
|
|
|
if (drop_struct_lock)
|
|
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
|
|
|
|
}
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
dnode_byteswap(dnode_phys_t *dnp)
|
|
|
|
|
{
|
|
|
|
|
uint64_t *buf64 = (void*)&dnp->dn_blkptr;
|
|
|
|
|
int i;
|
|
|
|
|
|
|
|
|
|
if (dnp->dn_type == DMU_OT_NONE) {
|
2022-02-25 16:26:54 +03:00
|
|
|
memset(dnp, 0, sizeof (dnode_phys_t));
|
2008-11-20 23:01:55 +03:00
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
dnp->dn_datablkszsec = BSWAP_16(dnp->dn_datablkszsec);
|
|
|
|
|
dnp->dn_bonuslen = BSWAP_16(dnp->dn_bonuslen);
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
dnp->dn_extra_slots = BSWAP_8(dnp->dn_extra_slots);
|
2008-11-20 23:01:55 +03:00
|
|
|
dnp->dn_maxblkid = BSWAP_64(dnp->dn_maxblkid);
|
|
|
|
|
dnp->dn_used = BSWAP_64(dnp->dn_used);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* dn_nblkptr is only one byte, so it's OK to read it in either
|
|
|
|
|
* byte order. We can't read dn_bouslen.
|
|
|
|
|
*/
|
|
|
|
|
ASSERT(dnp->dn_indblkshift <= SPA_MAXBLOCKSHIFT);
|
|
|
|
|
ASSERT(dnp->dn_nblkptr <= DN_MAX_NBLKPTR);
|
|
|
|
|
for (i = 0; i < dnp->dn_nblkptr * sizeof (blkptr_t)/8; i++)
|
|
|
|
|
buf64[i] = BSWAP_64(buf64[i]);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* OK to check dn_bonuslen for zero, because it won't matter if
|
|
|
|
|
* we have the wrong byte order. This is necessary because the
|
|
|
|
|
* dnode dnode is smaller than a regular dnode.
|
|
|
|
|
*/
|
|
|
|
|
if (dnp->dn_bonuslen != 0) {
|
2012-12-14 03:24:15 +04:00
|
|
|
dmu_object_byteswap_t byteswap;
|
|
|
|
|
ASSERT(DMU_OT_IS_VALID(dnp->dn_bonustype));
|
|
|
|
|
byteswap = DMU_OT_BYTESWAP(dnp->dn_bonustype);
|
2022-06-30 03:06:16 +03:00
|
|
|
dmu_ot_byteswap[byteswap].ob_func(DN_BONUS(dnp),
|
|
|
|
|
DN_MAX_BONUS_LEN(dnp));
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
2010-05-29 00:45:14 +04:00
|
|
|
|
|
|
|
|
/* Swap SPILL block if we have one */
|
|
|
|
|
if (dnp->dn_flags & DNODE_FLAG_SPILL_BLKPTR)
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
byteswap_uint64_array(DN_SPILL_BLKPTR(dnp), sizeof (blkptr_t));
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
dnode_buf_byteswap(void *vbuf, size_t size)
|
|
|
|
|
{
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
int i = 0;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
ASSERT3U(sizeof (dnode_phys_t), ==, (1<<DNODE_SHIFT));
|
|
|
|
|
ASSERT((size & (sizeof (dnode_phys_t)-1)) == 0);
|
|
|
|
|
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
while (i < size) {
|
2017-06-29 20:18:03 +03:00
|
|
|
dnode_phys_t *dnp = (void *)(((char *)vbuf) + i);
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
dnode_byteswap(dnp);
|
|
|
|
|
|
|
|
|
|
i += DNODE_MIN_SIZE;
|
|
|
|
|
if (dnp->dn_type != DMU_OT_NONE)
|
|
|
|
|
i += dnp->dn_extra_slots * DNODE_MIN_SIZE;
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
dnode_setbonuslen(dnode_t *dn, int newsize, dmu_tx_t *tx)
|
|
|
|
|
{
|
2018-10-01 20:42:05 +03:00
|
|
|
ASSERT3U(zfs_refcount_count(&dn->dn_holds), >=, 1);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
dnode_setdirty(dn, tx);
|
|
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_WRITER);
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
ASSERT3U(newsize, <=, DN_SLOTS_TO_BONUSLEN(dn->dn_num_slots) -
|
2008-11-20 23:01:55 +03:00
|
|
|
(dn->dn_nblkptr-1) * sizeof (blkptr_t));
|
2019-08-15 17:44:57 +03:00
|
|
|
|
|
|
|
|
if (newsize < dn->dn_bonuslen) {
|
|
|
|
|
/* clear any data after the end of the new size */
|
|
|
|
|
size_t diff = dn->dn_bonuslen - newsize;
|
|
|
|
|
char *data_end = ((char *)dn->dn_bonus->db.db_data) + newsize;
|
2022-02-25 16:26:54 +03:00
|
|
|
memset(data_end, 0, diff);
|
2019-08-15 17:44:57 +03:00
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_bonuslen = newsize;
|
|
|
|
|
if (newsize == 0)
|
|
|
|
|
dn->dn_next_bonuslen[tx->tx_txg & TXG_MASK] = DN_ZERO_BONUSLEN;
|
|
|
|
|
else
|
|
|
|
|
dn->dn_next_bonuslen[tx->tx_txg & TXG_MASK] = dn->dn_bonuslen;
|
|
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
|
|
|
|
}
|
|
|
|
|
|
2010-05-29 00:45:14 +04:00
|
|
|
void
|
|
|
|
|
dnode_setbonus_type(dnode_t *dn, dmu_object_type_t newtype, dmu_tx_t *tx)
|
|
|
|
|
{
|
2018-10-01 20:42:05 +03:00
|
|
|
ASSERT3U(zfs_refcount_count(&dn->dn_holds), >=, 1);
|
2010-05-29 00:45:14 +04:00
|
|
|
dnode_setdirty(dn, tx);
|
|
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_WRITER);
|
|
|
|
|
dn->dn_bonustype = newtype;
|
|
|
|
|
dn->dn_next_bonustype[tx->tx_txg & TXG_MASK] = dn->dn_bonustype;
|
|
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
dnode_rm_spill(dnode_t *dn, dmu_tx_t *tx)
|
|
|
|
|
{
|
2018-10-01 20:42:05 +03:00
|
|
|
ASSERT3U(zfs_refcount_count(&dn->dn_holds), >=, 1);
|
2010-05-29 00:45:14 +04:00
|
|
|
ASSERT(RW_WRITE_HELD(&dn->dn_struct_rwlock));
|
|
|
|
|
dnode_setdirty(dn, tx);
|
2019-04-12 21:30:59 +03:00
|
|
|
dn->dn_rm_spillblk[tx->tx_txg & TXG_MASK] = DN_KILL_SPILLBLK;
|
2010-05-29 00:45:14 +04:00
|
|
|
dn->dn_have_spill = B_FALSE;
|
|
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
static void
|
|
|
|
|
dnode_setdblksz(dnode_t *dn, int size)
|
|
|
|
|
{
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(P2PHASE(size, SPA_MINBLOCKSIZE));
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT3U(size, <=, SPA_MAXBLOCKSIZE);
|
|
|
|
|
ASSERT3U(size, >=, SPA_MINBLOCKSIZE);
|
|
|
|
|
ASSERT3U(size >> SPA_MINBLOCKSHIFT, <,
|
|
|
|
|
1<<(sizeof (dn->dn_phys->dn_datablkszsec) * 8));
|
|
|
|
|
dn->dn_datablksz = size;
|
|
|
|
|
dn->dn_datablkszsec = size >> SPA_MINBLOCKSHIFT;
|
2014-04-16 07:40:22 +04:00
|
|
|
dn->dn_datablkshift = ISP2(size) ? highbit64(size - 1) : 0;
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static dnode_t *
|
2017-09-06 02:15:04 +03:00
|
|
|
dnode_create(objset_t *os, dnode_phys_t *dnp, dmu_buf_impl_t *db,
|
2010-08-27 01:24:34 +04:00
|
|
|
uint64_t object, dnode_handle_t *dnh)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
2015-04-02 06:44:32 +03:00
|
|
|
dnode_t *dn;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2015-04-02 06:44:32 +03:00
|
|
|
dn = kmem_cache_alloc(dnode_cache, KM_SLEEP);
|
2010-08-27 01:24:34 +04:00
|
|
|
dn->dn_moved = 0;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Defer setting dn_objset until the dnode is ready to be a candidate
|
|
|
|
|
* for the dnode_move() callback.
|
|
|
|
|
*/
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_object = object;
|
|
|
|
|
dn->dn_dbuf = db;
|
2010-08-27 01:24:34 +04:00
|
|
|
dn->dn_handle = dnh;
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_phys = dnp;
|
|
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
if (dnp->dn_datablkszsec) {
|
2008-11-20 23:01:55 +03:00
|
|
|
dnode_setdblksz(dn, dnp->dn_datablkszsec << SPA_MINBLOCKSHIFT);
|
2010-08-27 01:24:34 +04:00
|
|
|
} else {
|
|
|
|
|
dn->dn_datablksz = 0;
|
|
|
|
|
dn->dn_datablkszsec = 0;
|
|
|
|
|
dn->dn_datablkshift = 0;
|
|
|
|
|
}
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_indblkshift = dnp->dn_indblkshift;
|
|
|
|
|
dn->dn_nlevels = dnp->dn_nlevels;
|
|
|
|
|
dn->dn_type = dnp->dn_type;
|
|
|
|
|
dn->dn_nblkptr = dnp->dn_nblkptr;
|
|
|
|
|
dn->dn_checksum = dnp->dn_checksum;
|
|
|
|
|
dn->dn_compress = dnp->dn_compress;
|
|
|
|
|
dn->dn_bonustype = dnp->dn_bonustype;
|
|
|
|
|
dn->dn_bonuslen = dnp->dn_bonuslen;
|
2017-09-06 02:15:04 +03:00
|
|
|
dn->dn_num_slots = dnp->dn_extra_slots + 1;
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_maxblkid = dnp->dn_maxblkid;
|
2010-05-29 00:45:14 +04:00
|
|
|
dn->dn_have_spill = ((dnp->dn_flags & DNODE_FLAG_SPILL_BLKPTR) != 0);
|
|
|
|
|
dn->dn_id_flags = 0;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
dmu_zfetch_init(&dn->dn_zfetch, dn);
|
|
|
|
|
|
2012-12-14 03:24:15 +04:00
|
|
|
ASSERT(DMU_OT_IS_VALID(dn->dn_phys->dn_type));
|
2017-09-06 02:15:04 +03:00
|
|
|
ASSERT(zrl_is_locked(&dnh->dnh_zrlock));
|
|
|
|
|
ASSERT(!DN_SLOT_IS_PTR(dnh->dnh_dnode));
|
2010-08-27 01:24:34 +04:00
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_enter(&os->os_lock);
|
2015-04-02 06:44:32 +03:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Exclude special dnodes from os_dnodes so an empty os_dnodes
|
|
|
|
|
* signifies that the special dnodes have no references from
|
|
|
|
|
* their children (the entries in os_dnodes). This allows
|
|
|
|
|
* dnode_destroy() to easily determine if the last child has
|
|
|
|
|
* been removed and then complete eviction of the objset.
|
|
|
|
|
*/
|
|
|
|
|
if (!DMU_OBJECT_IS_SPECIAL(object))
|
|
|
|
|
list_insert_head(&os->os_dnodes, dn);
|
2010-08-27 01:24:34 +04:00
|
|
|
membar_producer();
|
2015-04-02 06:44:32 +03:00
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
/*
|
2015-04-02 06:44:32 +03:00
|
|
|
* Everything else must be valid before assigning dn_objset
|
|
|
|
|
* makes the dnode eligible for dnode_move().
|
2010-08-27 01:24:34 +04:00
|
|
|
*/
|
|
|
|
|
dn->dn_objset = os;
|
2015-04-02 06:44:32 +03:00
|
|
|
|
|
|
|
|
dnh->dnh_dnode = dn;
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_exit(&os->os_lock);
|
|
|
|
|
|
2016-07-13 15:42:40 +03:00
|
|
|
arc_space_consume(sizeof (dnode_t), ARC_SPACE_DNODE);
|
2017-09-06 02:15:04 +03:00
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
return (dn);
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
/*
|
|
|
|
|
* Caller must be holding the dnode handle, which is released upon return.
|
|
|
|
|
*/
|
2008-11-20 23:01:55 +03:00
|
|
|
static void
|
|
|
|
|
dnode_destroy(dnode_t *dn)
|
|
|
|
|
{
|
2010-05-29 00:45:14 +04:00
|
|
|
objset_t *os = dn->dn_objset;
|
2015-04-02 06:44:32 +03:00
|
|
|
boolean_t complete_os_eviction = B_FALSE;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2010-05-29 00:45:14 +04:00
|
|
|
ASSERT((dn->dn_id_flags & DN_ID_NEW_EXIST) == 0);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
mutex_enter(&os->os_lock);
|
2010-08-27 01:24:34 +04:00
|
|
|
POINTER_INVALIDATE(&dn->dn_objset);
|
2015-04-02 06:44:32 +03:00
|
|
|
if (!DMU_OBJECT_IS_SPECIAL(dn->dn_object)) {
|
|
|
|
|
list_remove(&os->os_dnodes, dn);
|
|
|
|
|
complete_os_eviction =
|
|
|
|
|
list_is_empty(&os->os_dnodes) &&
|
|
|
|
|
list_link_active(&os->os_evicting_node);
|
|
|
|
|
}
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_exit(&os->os_lock);
|
|
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
/* the dnode can no longer move, so we can release the handle */
|
2018-01-19 12:19:47 +03:00
|
|
|
if (!zrl_is_locked(&dn->dn_handle->dnh_zrlock))
|
|
|
|
|
zrl_remove(&dn->dn_handle->dnh_zrlock);
|
2010-08-27 01:24:34 +04:00
|
|
|
|
|
|
|
|
dn->dn_allocated_txg = 0;
|
|
|
|
|
dn->dn_free_txg = 0;
|
|
|
|
|
dn->dn_assigned_txg = 0;
|
2018-04-10 21:15:05 +03:00
|
|
|
dn->dn_dirty_txg = 0;
|
2010-08-27 01:24:34 +04:00
|
|
|
|
|
|
|
|
dn->dn_dirtyctx = 0;
|
2020-02-27 03:09:17 +03:00
|
|
|
dn->dn_dirtyctx_firstset = NULL;
|
2010-08-27 01:24:34 +04:00
|
|
|
if (dn->dn_bonus != NULL) {
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_enter(&dn->dn_bonus->db_mtx);
|
2016-06-02 07:04:53 +03:00
|
|
|
dbuf_destroy(dn->dn_bonus);
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_bonus = NULL;
|
|
|
|
|
}
|
2010-08-27 01:24:34 +04:00
|
|
|
dn->dn_zio = NULL;
|
|
|
|
|
|
|
|
|
|
dn->dn_have_spill = B_FALSE;
|
|
|
|
|
dn->dn_oldused = 0;
|
|
|
|
|
dn->dn_oldflags = 0;
|
|
|
|
|
dn->dn_olduid = 0;
|
|
|
|
|
dn->dn_oldgid = 0;
|
2018-02-14 01:54:54 +03:00
|
|
|
dn->dn_oldprojid = ZFS_DEFAULT_PROJID;
|
2010-08-27 01:24:34 +04:00
|
|
|
dn->dn_newuid = 0;
|
|
|
|
|
dn->dn_newgid = 0;
|
2018-02-14 01:54:54 +03:00
|
|
|
dn->dn_newprojid = ZFS_DEFAULT_PROJID;
|
2010-08-27 01:24:34 +04:00
|
|
|
dn->dn_id_flags = 0;
|
|
|
|
|
|
2015-12-27 00:10:31 +03:00
|
|
|
dmu_zfetch_fini(&dn->dn_zfetch);
|
2008-11-20 23:01:55 +03:00
|
|
|
kmem_cache_free(dnode_cache, dn);
|
2016-07-13 15:42:40 +03:00
|
|
|
arc_space_return(sizeof (dnode_t), ARC_SPACE_DNODE);
|
2015-04-02 06:44:32 +03:00
|
|
|
|
|
|
|
|
if (complete_os_eviction)
|
|
|
|
|
dmu_objset_evict_done(os);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
dnode_allocate(dnode_t *dn, dmu_object_type_t ot, int blocksize, int ibs,
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
dmu_object_type_t bonustype, int bonuslen, int dn_slots, dmu_tx_t *tx)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
|
|
|
|
int i;
|
|
|
|
|
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
ASSERT3U(dn_slots, >, 0);
|
|
|
|
|
ASSERT3U(dn_slots << DNODE_SHIFT, <=,
|
|
|
|
|
spa_maxdnodesize(dmu_objset_spa(dn->dn_objset)));
|
2014-11-03 23:15:08 +03:00
|
|
|
ASSERT3U(blocksize, <=,
|
|
|
|
|
spa_maxblocksize(dmu_objset_spa(dn->dn_objset)));
|
2008-11-20 23:01:55 +03:00
|
|
|
if (blocksize == 0)
|
|
|
|
|
blocksize = 1 << zfs_default_bs;
|
|
|
|
|
else
|
|
|
|
|
blocksize = P2ROUNDUP(blocksize, SPA_MINBLOCKSIZE);
|
|
|
|
|
|
|
|
|
|
if (ibs == 0)
|
|
|
|
|
ibs = zfs_default_ibs;
|
|
|
|
|
|
|
|
|
|
ibs = MIN(MAX(ibs, DN_MIN_INDBLKSHIFT), DN_MAX_INDBLKSHIFT);
|
|
|
|
|
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
dprintf("os=%p obj=%llu txg=%llu blocksize=%d ibs=%d dn_slots=%d\n",
|
2021-06-23 07:53:45 +03:00
|
|
|
dn->dn_objset, (u_longlong_t)dn->dn_object,
|
|
|
|
|
(u_longlong_t)tx->tx_txg, blocksize, ibs, dn_slots);
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_allocate);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
ASSERT(dn->dn_type == DMU_OT_NONE);
|
2022-02-25 16:26:54 +03:00
|
|
|
ASSERT0(memcmp(dn->dn_phys, &dnode_phys_zero, sizeof (dnode_phys_t)));
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT(dn->dn_phys->dn_type == DMU_OT_NONE);
|
|
|
|
|
ASSERT(ot != DMU_OT_NONE);
|
2012-12-14 03:24:15 +04:00
|
|
|
ASSERT(DMU_OT_IS_VALID(ot));
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT((bonustype == DMU_OT_NONE && bonuslen == 0) ||
|
2010-05-29 00:45:14 +04:00
|
|
|
(bonustype == DMU_OT_SA && bonuslen == 0) ||
|
2008-11-20 23:01:55 +03:00
|
|
|
(bonustype != DMU_OT_NONE && bonuslen != 0));
|
2012-12-14 03:24:15 +04:00
|
|
|
ASSERT(DMU_OT_IS_VALID(bonustype));
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
ASSERT3U(bonuslen, <=, DN_SLOTS_TO_BONUSLEN(dn_slots));
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT(dn->dn_type == DMU_OT_NONE);
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(dn->dn_maxblkid);
|
|
|
|
|
ASSERT0(dn->dn_allocated_txg);
|
|
|
|
|
ASSERT0(dn->dn_assigned_txg);
|
2018-10-01 20:42:05 +03:00
|
|
|
ASSERT(zfs_refcount_is_zero(&dn->dn_tx_holds));
|
|
|
|
|
ASSERT3U(zfs_refcount_count(&dn->dn_holds), <=, 1);
|
2015-04-03 06:14:28 +03:00
|
|
|
ASSERT(avl_is_empty(&dn->dn_dbufs));
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
for (i = 0; i < TXG_SIZE; i++) {
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(dn->dn_next_nblkptr[i]);
|
|
|
|
|
ASSERT0(dn->dn_next_nlevels[i]);
|
|
|
|
|
ASSERT0(dn->dn_next_indblkshift[i]);
|
|
|
|
|
ASSERT0(dn->dn_next_bonuslen[i]);
|
|
|
|
|
ASSERT0(dn->dn_next_bonustype[i]);
|
|
|
|
|
ASSERT0(dn->dn_rm_spillblk[i]);
|
|
|
|
|
ASSERT0(dn->dn_next_blksz[i]);
|
2017-11-08 22:12:59 +03:00
|
|
|
ASSERT0(dn->dn_next_maxblkid[i]);
|
2018-04-10 21:15:05 +03:00
|
|
|
ASSERT(!multilist_link_active(&dn->dn_dirty_link[i]));
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT3P(list_head(&dn->dn_dirty_records[i]), ==, NULL);
|
2014-04-16 07:40:22 +04:00
|
|
|
ASSERT3P(dn->dn_free_ranges[i], ==, NULL);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
dn->dn_type = ot;
|
|
|
|
|
dnode_setdblksz(dn, blocksize);
|
|
|
|
|
dn->dn_indblkshift = ibs;
|
|
|
|
|
dn->dn_nlevels = 1;
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
dn->dn_num_slots = dn_slots;
|
2010-05-29 00:45:14 +04:00
|
|
|
if (bonustype == DMU_OT_SA) /* Maximize bonus space for SA */
|
|
|
|
|
dn->dn_nblkptr = 1;
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
else {
|
|
|
|
|
dn->dn_nblkptr = MIN(DN_MAX_NBLKPTR,
|
|
|
|
|
1 + ((DN_SLOTS_TO_BONUSLEN(dn_slots) - bonuslen) >>
|
|
|
|
|
SPA_BLKPTRSHIFT));
|
|
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_bonustype = bonustype;
|
|
|
|
|
dn->dn_bonuslen = bonuslen;
|
|
|
|
|
dn->dn_checksum = ZIO_CHECKSUM_INHERIT;
|
|
|
|
|
dn->dn_compress = ZIO_COMPRESS_INHERIT;
|
|
|
|
|
dn->dn_dirtyctx = 0;
|
|
|
|
|
|
|
|
|
|
dn->dn_free_txg = 0;
|
2020-02-27 03:09:17 +03:00
|
|
|
dn->dn_dirtyctx_firstset = NULL;
|
Improve zfs receive performance with lightweight write
The performance of `zfs receive` can be bottlenecked on the CPU consumed
by the `receive_writer` thread, especially when receiving streams with
small compressed block sizes. Much of the CPU is spent creating and
destroying dbuf's and arc buf's, one for each `WRITE` record in the send
stream.
This commit introduces the concept of "lightweight writes", which allows
`zfs receive` to write to the DMU by providing an ABD, and instantiating
only a new type of `dbuf_dirty_record_t`. The dbuf and arc buf for this
"dirty leaf block" are not instantiated.
Because there is no dbuf with the dirty data, this mechanism doesn't
support reading from "lightweight-dirty" blocks (they would see the
on-disk state rather than the dirty data). Since the dedup-receive code
has been removed, `zfs receive` is write-only, so this works fine.
Because there are no arc bufs for the received data, the received data
is no longer cached in the ARC.
Testing a receive of a stream with average compressed block size of 4KB,
this commit improves performance by 50%, while also reducing CPU usage
by 50% of a CPU. On a per-block basis, CPU consumed by receive_writer()
and dbuf_evict() is now 1/7th (14%) of what it was.
Baseline: 450MB/s, CPU in receive_writer() 40% + dbuf_evict() 35%
New: 670MB/s, CPU in receive_writer() 17% + dbuf_evict() 0%
The code is also restructured in a few ways:
Added a `dr_dnode` field to the dbuf_dirty_record_t. This simplifies
some existing code that no longer needs `DB_DNODE_ENTER()` and related
routines. The new field is needed by the lightweight-type dirty record.
To ensure that the `dr_dnode` field remains valid until the dirty record
is freed, we have to ensure that the `dnode_move()` doesn't relocate the
dnode_t. To do this we keep a hold on the dnode until it's zio's have
completed. This is already done by the user-accounting code
(`userquota_updates_task()`), this commit extends that so that it always
keeps the dnode hold until zio completion (see `dnode_rele_task()`).
`dn_dirty_txg` was previously zeroed when the dnode was synced. This
was not necessary, since its meaning can be "when was this dnode last
dirtied". This change simplifies the new `dnode_rele_task()` code.
Removed some dead code related to `DRR_WRITE_BYREF` (dedup receive).
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #11105
2020-12-11 21:26:02 +03:00
|
|
|
dn->dn_dirty_txg = 0;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
dn->dn_allocated_txg = tx->tx_txg;
|
2010-05-29 00:45:14 +04:00
|
|
|
dn->dn_id_flags = 0;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
dnode_setdirty(dn, tx);
|
|
|
|
|
dn->dn_next_indblkshift[tx->tx_txg & TXG_MASK] = ibs;
|
|
|
|
|
dn->dn_next_bonuslen[tx->tx_txg & TXG_MASK] = dn->dn_bonuslen;
|
2010-05-29 00:45:14 +04:00
|
|
|
dn->dn_next_bonustype[tx->tx_txg & TXG_MASK] = dn->dn_bonustype;
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_next_blksz[tx->tx_txg & TXG_MASK] = dn->dn_datablksz;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
dnode_reallocate(dnode_t *dn, dmu_object_type_t ot, int blocksize,
|
2019-05-08 01:18:44 +03:00
|
|
|
dmu_object_type_t bonustype, int bonuslen, int dn_slots,
|
|
|
|
|
boolean_t keep_spill, dmu_tx_t *tx)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
2009-07-03 02:44:48 +04:00
|
|
|
int nblkptr;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
ASSERT3U(blocksize, >=, SPA_MINBLOCKSIZE);
|
2014-11-03 23:15:08 +03:00
|
|
|
ASSERT3U(blocksize, <=,
|
|
|
|
|
spa_maxblocksize(dmu_objset_spa(dn->dn_objset)));
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(blocksize % SPA_MINBLOCKSIZE);
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT(dn->dn_object != DMU_META_DNODE_OBJECT || dmu_tx_private_ok(tx));
|
|
|
|
|
ASSERT(tx->tx_txg != 0);
|
|
|
|
|
ASSERT((bonustype == DMU_OT_NONE && bonuslen == 0) ||
|
2010-05-29 00:45:14 +04:00
|
|
|
(bonustype != DMU_OT_NONE && bonuslen != 0) ||
|
|
|
|
|
(bonustype == DMU_OT_SA && bonuslen == 0));
|
2012-12-14 03:24:15 +04:00
|
|
|
ASSERT(DMU_OT_IS_VALID(bonustype));
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
ASSERT3U(bonuslen, <=,
|
2016-12-12 21:46:26 +03:00
|
|
|
DN_BONUS_SIZE(spa_maxdnodesize(dmu_objset_spa(dn->dn_objset))));
|
2018-04-17 21:13:57 +03:00
|
|
|
ASSERT3U(bonuslen, <=, DN_BONUS_SIZE(dn_slots << DNODE_SHIFT));
|
2018-01-19 12:19:47 +03:00
|
|
|
|
|
|
|
|
dnode_free_interior_slots(dn);
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_reallocate);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
/* clean up any unreferenced dbufs */
|
|
|
|
|
dnode_evict_dbufs(dn);
|
2009-02-18 23:51:31 +03:00
|
|
|
|
2010-05-29 00:45:14 +04:00
|
|
|
dn->dn_id_flags = 0;
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_WRITER);
|
|
|
|
|
dnode_setdirty(dn, tx);
|
2009-07-03 02:44:48 +04:00
|
|
|
if (dn->dn_datablksz != blocksize) {
|
2019-04-06 03:32:56 +03:00
|
|
|
/* change blocksize */
|
2019-04-12 21:30:59 +03:00
|
|
|
ASSERT0(dn->dn_maxblkid);
|
|
|
|
|
ASSERT(BP_IS_HOLE(&dn->dn_phys->dn_blkptr[0]) ||
|
|
|
|
|
dnode_block_freed(dn, 0));
|
|
|
|
|
|
2019-04-06 03:32:56 +03:00
|
|
|
dnode_setdblksz(dn, blocksize);
|
2019-04-12 21:30:59 +03:00
|
|
|
dn->dn_next_blksz[tx->tx_txg & TXG_MASK] = blocksize;
|
2009-07-03 02:44:48 +04:00
|
|
|
}
|
|
|
|
|
if (dn->dn_bonuslen != bonuslen)
|
2019-04-12 21:30:59 +03:00
|
|
|
dn->dn_next_bonuslen[tx->tx_txg & TXG_MASK] = bonuslen;
|
2010-05-29 00:45:14 +04:00
|
|
|
|
|
|
|
|
if (bonustype == DMU_OT_SA) /* Maximize bonus space for SA */
|
|
|
|
|
nblkptr = 1;
|
|
|
|
|
else
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
nblkptr = MIN(DN_MAX_NBLKPTR,
|
|
|
|
|
1 + ((DN_SLOTS_TO_BONUSLEN(dn_slots) - bonuslen) >>
|
|
|
|
|
SPA_BLKPTRSHIFT));
|
2010-05-29 00:45:14 +04:00
|
|
|
if (dn->dn_bonustype != bonustype)
|
2019-04-12 21:30:59 +03:00
|
|
|
dn->dn_next_bonustype[tx->tx_txg & TXG_MASK] = bonustype;
|
2009-02-18 23:51:31 +03:00
|
|
|
if (dn->dn_nblkptr != nblkptr)
|
2019-04-12 21:30:59 +03:00
|
|
|
dn->dn_next_nblkptr[tx->tx_txg & TXG_MASK] = nblkptr;
|
2019-05-08 01:18:44 +03:00
|
|
|
if (dn->dn_phys->dn_flags & DNODE_FLAG_SPILL_BLKPTR && !keep_spill) {
|
2010-05-29 00:45:14 +04:00
|
|
|
dbuf_rm_spill(dn, tx);
|
|
|
|
|
dnode_rm_spill(dn, tx);
|
|
|
|
|
}
|
2019-04-12 21:30:59 +03:00
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
|
|
|
|
|
|
|
|
|
/* change type */
|
|
|
|
|
dn->dn_type = ot;
|
|
|
|
|
|
|
|
|
|
/* change bonus size and type */
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
|
|
|
|
dn->dn_bonustype = bonustype;
|
|
|
|
|
dn->dn_bonuslen = bonuslen;
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
dn->dn_num_slots = dn_slots;
|
2009-02-18 23:51:31 +03:00
|
|
|
dn->dn_nblkptr = nblkptr;
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_checksum = ZIO_CHECKSUM_INHERIT;
|
|
|
|
|
dn->dn_compress = ZIO_COMPRESS_INHERIT;
|
|
|
|
|
ASSERT3U(dn->dn_nblkptr, <=, DN_MAX_NBLKPTR);
|
|
|
|
|
|
2009-02-18 23:51:31 +03:00
|
|
|
/* fix up the bonus db_size */
|
|
|
|
|
if (dn->dn_bonus) {
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_bonus->db.db_size =
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
DN_SLOTS_TO_BONUSLEN(dn->dn_num_slots) -
|
|
|
|
|
(dn->dn_nblkptr-1) * sizeof (blkptr_t);
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT(dn->dn_bonuslen <= dn->dn_bonus->db.db_size);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
dn->dn_allocated_txg = tx->tx_txg;
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
}
|
|
|
|
|
|
2010-08-27 21:55:07 +04:00
|
|
|
#ifdef _KERNEL
|
2010-08-27 01:24:34 +04:00
|
|
|
static void
|
|
|
|
|
dnode_move_impl(dnode_t *odn, dnode_t *ndn)
|
|
|
|
|
{
|
|
|
|
|
ASSERT(!RW_LOCK_HELD(&odn->dn_struct_rwlock));
|
|
|
|
|
ASSERT(MUTEX_NOT_HELD(&odn->dn_mtx));
|
|
|
|
|
ASSERT(MUTEX_NOT_HELD(&odn->dn_dbufs_mtx));
|
|
|
|
|
|
|
|
|
|
/* Copy fields. */
|
|
|
|
|
ndn->dn_objset = odn->dn_objset;
|
|
|
|
|
ndn->dn_object = odn->dn_object;
|
|
|
|
|
ndn->dn_dbuf = odn->dn_dbuf;
|
|
|
|
|
ndn->dn_handle = odn->dn_handle;
|
|
|
|
|
ndn->dn_phys = odn->dn_phys;
|
|
|
|
|
ndn->dn_type = odn->dn_type;
|
|
|
|
|
ndn->dn_bonuslen = odn->dn_bonuslen;
|
|
|
|
|
ndn->dn_bonustype = odn->dn_bonustype;
|
|
|
|
|
ndn->dn_nblkptr = odn->dn_nblkptr;
|
|
|
|
|
ndn->dn_checksum = odn->dn_checksum;
|
|
|
|
|
ndn->dn_compress = odn->dn_compress;
|
|
|
|
|
ndn->dn_nlevels = odn->dn_nlevels;
|
|
|
|
|
ndn->dn_indblkshift = odn->dn_indblkshift;
|
|
|
|
|
ndn->dn_datablkshift = odn->dn_datablkshift;
|
|
|
|
|
ndn->dn_datablkszsec = odn->dn_datablkszsec;
|
|
|
|
|
ndn->dn_datablksz = odn->dn_datablksz;
|
|
|
|
|
ndn->dn_maxblkid = odn->dn_maxblkid;
|
2017-06-29 20:18:03 +03:00
|
|
|
ndn->dn_num_slots = odn->dn_num_slots;
|
2022-02-25 16:26:54 +03:00
|
|
|
memcpy(ndn->dn_next_type, odn->dn_next_type,
|
2017-06-23 22:32:16 +03:00
|
|
|
sizeof (odn->dn_next_type));
|
2022-02-25 16:26:54 +03:00
|
|
|
memcpy(ndn->dn_next_nblkptr, odn->dn_next_nblkptr,
|
2010-08-27 01:24:34 +04:00
|
|
|
sizeof (odn->dn_next_nblkptr));
|
2022-02-25 16:26:54 +03:00
|
|
|
memcpy(ndn->dn_next_nlevels, odn->dn_next_nlevels,
|
2010-08-27 01:24:34 +04:00
|
|
|
sizeof (odn->dn_next_nlevels));
|
2022-02-25 16:26:54 +03:00
|
|
|
memcpy(ndn->dn_next_indblkshift, odn->dn_next_indblkshift,
|
2010-08-27 01:24:34 +04:00
|
|
|
sizeof (odn->dn_next_indblkshift));
|
2022-02-25 16:26:54 +03:00
|
|
|
memcpy(ndn->dn_next_bonustype, odn->dn_next_bonustype,
|
2010-08-27 01:24:34 +04:00
|
|
|
sizeof (odn->dn_next_bonustype));
|
2022-02-25 16:26:54 +03:00
|
|
|
memcpy(ndn->dn_rm_spillblk, odn->dn_rm_spillblk,
|
2010-08-27 01:24:34 +04:00
|
|
|
sizeof (odn->dn_rm_spillblk));
|
2022-02-25 16:26:54 +03:00
|
|
|
memcpy(ndn->dn_next_bonuslen, odn->dn_next_bonuslen,
|
2010-08-27 01:24:34 +04:00
|
|
|
sizeof (odn->dn_next_bonuslen));
|
2022-02-25 16:26:54 +03:00
|
|
|
memcpy(ndn->dn_next_blksz, odn->dn_next_blksz,
|
2010-08-27 01:24:34 +04:00
|
|
|
sizeof (odn->dn_next_blksz));
|
2022-02-25 16:26:54 +03:00
|
|
|
memcpy(ndn->dn_next_maxblkid, odn->dn_next_maxblkid,
|
2017-11-08 22:12:59 +03:00
|
|
|
sizeof (odn->dn_next_maxblkid));
|
2022-02-25 16:26:54 +03:00
|
|
|
for (int i = 0; i < TXG_SIZE; i++) {
|
2010-08-27 01:24:34 +04:00
|
|
|
list_move_tail(&ndn->dn_dirty_records[i],
|
|
|
|
|
&odn->dn_dirty_records[i]);
|
|
|
|
|
}
|
2022-02-25 16:26:54 +03:00
|
|
|
memcpy(ndn->dn_free_ranges, odn->dn_free_ranges,
|
2014-04-16 07:40:22 +04:00
|
|
|
sizeof (odn->dn_free_ranges));
|
2010-08-27 01:24:34 +04:00
|
|
|
ndn->dn_allocated_txg = odn->dn_allocated_txg;
|
|
|
|
|
ndn->dn_free_txg = odn->dn_free_txg;
|
|
|
|
|
ndn->dn_assigned_txg = odn->dn_assigned_txg;
|
2018-04-10 21:15:05 +03:00
|
|
|
ndn->dn_dirty_txg = odn->dn_dirty_txg;
|
2010-08-27 01:24:34 +04:00
|
|
|
ndn->dn_dirtyctx = odn->dn_dirtyctx;
|
|
|
|
|
ndn->dn_dirtyctx_firstset = odn->dn_dirtyctx_firstset;
|
2018-10-01 20:42:05 +03:00
|
|
|
ASSERT(zfs_refcount_count(&odn->dn_tx_holds) == 0);
|
|
|
|
|
zfs_refcount_transfer(&ndn->dn_holds, &odn->dn_holds);
|
2015-04-03 06:14:28 +03:00
|
|
|
ASSERT(avl_is_empty(&ndn->dn_dbufs));
|
|
|
|
|
avl_swap(&ndn->dn_dbufs, &odn->dn_dbufs);
|
2010-08-27 01:24:34 +04:00
|
|
|
ndn->dn_dbufs_count = odn->dn_dbufs_count;
|
|
|
|
|
ndn->dn_bonus = odn->dn_bonus;
|
|
|
|
|
ndn->dn_have_spill = odn->dn_have_spill;
|
|
|
|
|
ndn->dn_zio = odn->dn_zio;
|
|
|
|
|
ndn->dn_oldused = odn->dn_oldused;
|
|
|
|
|
ndn->dn_oldflags = odn->dn_oldflags;
|
|
|
|
|
ndn->dn_olduid = odn->dn_olduid;
|
|
|
|
|
ndn->dn_oldgid = odn->dn_oldgid;
|
2018-02-14 01:54:54 +03:00
|
|
|
ndn->dn_oldprojid = odn->dn_oldprojid;
|
2010-08-27 01:24:34 +04:00
|
|
|
ndn->dn_newuid = odn->dn_newuid;
|
|
|
|
|
ndn->dn_newgid = odn->dn_newgid;
|
2018-02-14 01:54:54 +03:00
|
|
|
ndn->dn_newprojid = odn->dn_newprojid;
|
2010-08-27 01:24:34 +04:00
|
|
|
ndn->dn_id_flags = odn->dn_id_flags;
|
2021-05-08 01:07:03 +03:00
|
|
|
dmu_zfetch_init(&ndn->dn_zfetch, ndn);
|
2010-08-27 01:24:34 +04:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Update back pointers. Updating the handle fixes the back pointer of
|
|
|
|
|
* every descendant dbuf as well as the bonus dbuf.
|
|
|
|
|
*/
|
|
|
|
|
ASSERT(ndn->dn_handle->dnh_dnode == odn);
|
|
|
|
|
ndn->dn_handle->dnh_dnode = ndn;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Invalidate the original dnode by clearing all of its back pointers.
|
|
|
|
|
*/
|
|
|
|
|
odn->dn_dbuf = NULL;
|
|
|
|
|
odn->dn_handle = NULL;
|
2015-04-03 06:14:28 +03:00
|
|
|
avl_create(&odn->dn_dbufs, dbuf_compare, sizeof (dmu_buf_impl_t),
|
2010-08-27 01:24:34 +04:00
|
|
|
offsetof(dmu_buf_impl_t, db_link));
|
|
|
|
|
odn->dn_dbufs_count = 0;
|
|
|
|
|
odn->dn_bonus = NULL;
|
2018-11-30 05:20:44 +03:00
|
|
|
dmu_zfetch_fini(&odn->dn_zfetch);
|
2010-08-27 01:24:34 +04:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Set the low bit of the objset pointer to ensure that dnode_move()
|
|
|
|
|
* recognizes the dnode as invalid in any subsequent callback.
|
|
|
|
|
*/
|
|
|
|
|
POINTER_INVALIDATE(&odn->dn_objset);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Satisfy the destructor.
|
|
|
|
|
*/
|
2022-02-25 16:26:54 +03:00
|
|
|
for (int i = 0; i < TXG_SIZE; i++) {
|
2010-08-27 01:24:34 +04:00
|
|
|
list_create(&odn->dn_dirty_records[i],
|
|
|
|
|
sizeof (dbuf_dirty_record_t),
|
|
|
|
|
offsetof(dbuf_dirty_record_t, dr_dirty_node));
|
2014-04-16 07:40:22 +04:00
|
|
|
odn->dn_free_ranges[i] = NULL;
|
2010-08-27 01:24:34 +04:00
|
|
|
odn->dn_next_nlevels[i] = 0;
|
|
|
|
|
odn->dn_next_indblkshift[i] = 0;
|
|
|
|
|
odn->dn_next_bonustype[i] = 0;
|
|
|
|
|
odn->dn_rm_spillblk[i] = 0;
|
|
|
|
|
odn->dn_next_bonuslen[i] = 0;
|
|
|
|
|
odn->dn_next_blksz[i] = 0;
|
|
|
|
|
}
|
|
|
|
|
odn->dn_allocated_txg = 0;
|
|
|
|
|
odn->dn_free_txg = 0;
|
|
|
|
|
odn->dn_assigned_txg = 0;
|
2018-04-10 21:15:05 +03:00
|
|
|
odn->dn_dirty_txg = 0;
|
2010-08-27 01:24:34 +04:00
|
|
|
odn->dn_dirtyctx = 0;
|
|
|
|
|
odn->dn_dirtyctx_firstset = NULL;
|
|
|
|
|
odn->dn_have_spill = B_FALSE;
|
|
|
|
|
odn->dn_zio = NULL;
|
|
|
|
|
odn->dn_oldused = 0;
|
|
|
|
|
odn->dn_oldflags = 0;
|
|
|
|
|
odn->dn_olduid = 0;
|
|
|
|
|
odn->dn_oldgid = 0;
|
2018-02-14 01:54:54 +03:00
|
|
|
odn->dn_oldprojid = ZFS_DEFAULT_PROJID;
|
2010-08-27 01:24:34 +04:00
|
|
|
odn->dn_newuid = 0;
|
|
|
|
|
odn->dn_newgid = 0;
|
2018-02-14 01:54:54 +03:00
|
|
|
odn->dn_newprojid = ZFS_DEFAULT_PROJID;
|
2010-08-27 01:24:34 +04:00
|
|
|
odn->dn_id_flags = 0;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Mark the dnode.
|
|
|
|
|
*/
|
|
|
|
|
ndn->dn_moved = 1;
|
|
|
|
|
odn->dn_moved = (uint8_t)-1;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static kmem_cbrc_t
|
|
|
|
|
dnode_move(void *buf, void *newbuf, size_t size, void *arg)
|
|
|
|
|
{
|
|
|
|
|
dnode_t *odn = buf, *ndn = newbuf;
|
|
|
|
|
objset_t *os;
|
|
|
|
|
int64_t refcount;
|
|
|
|
|
uint32_t dbufs;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The dnode is on the objset's list of known dnodes if the objset
|
|
|
|
|
* pointer is valid. We set the low bit of the objset pointer when
|
|
|
|
|
* freeing the dnode to invalidate it, and the memory patterns written
|
|
|
|
|
* by kmem (baddcafe and deadbeef) set at least one of the two low bits.
|
|
|
|
|
* A newly created dnode sets the objset pointer last of all to indicate
|
|
|
|
|
* that the dnode is known and in a valid state to be moved by this
|
|
|
|
|
* function.
|
|
|
|
|
*/
|
|
|
|
|
os = odn->dn_objset;
|
|
|
|
|
if (!POINTER_IS_VALID(os)) {
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_move_invalid);
|
2010-08-27 01:24:34 +04:00
|
|
|
return (KMEM_CBRC_DONT_KNOW);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Ensure that the objset does not go away during the move.
|
|
|
|
|
*/
|
|
|
|
|
rw_enter(&os_lock, RW_WRITER);
|
|
|
|
|
if (os != odn->dn_objset) {
|
|
|
|
|
rw_exit(&os_lock);
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_move_recheck1);
|
2010-08-27 01:24:34 +04:00
|
|
|
return (KMEM_CBRC_DONT_KNOW);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If the dnode is still valid, then so is the objset. We know that no
|
|
|
|
|
* valid objset can be freed while we hold os_lock, so we can safely
|
|
|
|
|
* ensure that the objset remains in use.
|
|
|
|
|
*/
|
|
|
|
|
mutex_enter(&os->os_lock);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Recheck the objset pointer in case the dnode was removed just before
|
|
|
|
|
* acquiring the lock.
|
|
|
|
|
*/
|
|
|
|
|
if (os != odn->dn_objset) {
|
|
|
|
|
mutex_exit(&os->os_lock);
|
|
|
|
|
rw_exit(&os_lock);
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_move_recheck2);
|
2010-08-27 01:24:34 +04:00
|
|
|
return (KMEM_CBRC_DONT_KNOW);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* At this point we know that as long as we hold os->os_lock, the dnode
|
|
|
|
|
* cannot be freed and fields within the dnode can be safely accessed.
|
|
|
|
|
* The objset listing this dnode cannot go away as long as this dnode is
|
|
|
|
|
* on its list.
|
|
|
|
|
*/
|
|
|
|
|
rw_exit(&os_lock);
|
|
|
|
|
if (DMU_OBJECT_IS_SPECIAL(odn->dn_object)) {
|
|
|
|
|
mutex_exit(&os->os_lock);
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_move_special);
|
2010-08-27 01:24:34 +04:00
|
|
|
return (KMEM_CBRC_NO);
|
|
|
|
|
}
|
|
|
|
|
ASSERT(odn->dn_dbuf != NULL); /* only "special" dnodes have no parent */
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Lock the dnode handle to prevent the dnode from obtaining any new
|
|
|
|
|
* holds. This also prevents the descendant dbufs and the bonus dbuf
|
|
|
|
|
* from accessing the dnode, so that we can discount their holds. The
|
|
|
|
|
* handle is safe to access because we know that while the dnode cannot
|
|
|
|
|
* go away, neither can its handle. Once we hold dnh_zrlock, we can
|
|
|
|
|
* safely move any dnode referenced only by dbufs.
|
|
|
|
|
*/
|
|
|
|
|
if (!zrl_tryenter(&odn->dn_handle->dnh_zrlock)) {
|
|
|
|
|
mutex_exit(&os->os_lock);
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_move_handle);
|
2010-08-27 01:24:34 +04:00
|
|
|
return (KMEM_CBRC_LATER);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Ensure a consistent view of the dnode's holds and the dnode's dbufs.
|
|
|
|
|
* We need to guarantee that there is a hold for every dbuf in order to
|
|
|
|
|
* determine whether the dnode is actively referenced. Falsely matching
|
|
|
|
|
* a dbuf to an active hold would lead to an unsafe move. It's possible
|
|
|
|
|
* that a thread already having an active dnode hold is about to add a
|
|
|
|
|
* dbuf, and we can't compare hold and dbuf counts while the add is in
|
|
|
|
|
* progress.
|
|
|
|
|
*/
|
|
|
|
|
if (!rw_tryenter(&odn->dn_struct_rwlock, RW_WRITER)) {
|
|
|
|
|
zrl_exit(&odn->dn_handle->dnh_zrlock);
|
|
|
|
|
mutex_exit(&os->os_lock);
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_move_rwlock);
|
2010-08-27 01:24:34 +04:00
|
|
|
return (KMEM_CBRC_LATER);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* A dbuf may be removed (evicted) without an active dnode hold. In that
|
|
|
|
|
* case, the dbuf count is decremented under the handle lock before the
|
|
|
|
|
* dbuf's hold is released. This order ensures that if we count the hold
|
|
|
|
|
* after the dbuf is removed but before its hold is released, we will
|
|
|
|
|
* treat the unmatched hold as active and exit safely. If we count the
|
|
|
|
|
* hold before the dbuf is removed, the hold is discounted, and the
|
|
|
|
|
* removal is blocked until the move completes.
|
|
|
|
|
*/
|
2018-10-01 20:42:05 +03:00
|
|
|
refcount = zfs_refcount_count(&odn->dn_holds);
|
2010-08-27 01:24:34 +04:00
|
|
|
ASSERT(refcount >= 0);
|
2020-02-13 22:20:42 +03:00
|
|
|
dbufs = DN_DBUFS_COUNT(odn);
|
2010-08-27 01:24:34 +04:00
|
|
|
|
|
|
|
|
/* We can't have more dbufs than dnode holds. */
|
|
|
|
|
ASSERT3U(dbufs, <=, refcount);
|
|
|
|
|
DTRACE_PROBE3(dnode__move, dnode_t *, odn, int64_t, refcount,
|
|
|
|
|
uint32_t, dbufs);
|
|
|
|
|
|
|
|
|
|
if (refcount > dbufs) {
|
|
|
|
|
rw_exit(&odn->dn_struct_rwlock);
|
|
|
|
|
zrl_exit(&odn->dn_handle->dnh_zrlock);
|
|
|
|
|
mutex_exit(&os->os_lock);
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_move_active);
|
2010-08-27 01:24:34 +04:00
|
|
|
return (KMEM_CBRC_LATER);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
rw_exit(&odn->dn_struct_rwlock);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* At this point we know that anyone with a hold on the dnode is not
|
|
|
|
|
* actively referencing it. The dnode is known and in a valid state to
|
|
|
|
|
* move. We're holding the locks needed to execute the critical section.
|
|
|
|
|
*/
|
|
|
|
|
dnode_move_impl(odn, ndn);
|
|
|
|
|
|
|
|
|
|
list_link_replace(&odn->dn_link, &ndn->dn_link);
|
|
|
|
|
/* If the dnode was safe to move, the refcount cannot have changed. */
|
2018-10-01 20:42:05 +03:00
|
|
|
ASSERT(refcount == zfs_refcount_count(&ndn->dn_holds));
|
2020-02-13 22:20:42 +03:00
|
|
|
ASSERT(dbufs == DN_DBUFS_COUNT(ndn));
|
2010-08-27 01:24:34 +04:00
|
|
|
zrl_exit(&ndn->dn_handle->dnh_zrlock); /* handle has moved */
|
|
|
|
|
mutex_exit(&os->os_lock);
|
|
|
|
|
|
|
|
|
|
return (KMEM_CBRC_YES);
|
|
|
|
|
}
|
|
|
|
|
#endif /* _KERNEL */
|
|
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
static void
|
|
|
|
|
dnode_slots_hold(dnode_children_t *children, int idx, int slots)
|
|
|
|
|
{
|
|
|
|
|
ASSERT3S(idx + slots, <=, DNODES_PER_BLOCK);
|
|
|
|
|
|
|
|
|
|
for (int i = idx; i < idx + slots; i++) {
|
|
|
|
|
dnode_handle_t *dnh = &children->dnc_children[i];
|
|
|
|
|
zrl_add(&dnh->dnh_zrlock);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void
|
|
|
|
|
dnode_slots_rele(dnode_children_t *children, int idx, int slots)
|
|
|
|
|
{
|
|
|
|
|
ASSERT3S(idx + slots, <=, DNODES_PER_BLOCK);
|
|
|
|
|
|
|
|
|
|
for (int i = idx; i < idx + slots; i++) {
|
|
|
|
|
dnode_handle_t *dnh = &children->dnc_children[i];
|
|
|
|
|
|
|
|
|
|
if (zrl_is_locked(&dnh->dnh_zrlock))
|
|
|
|
|
zrl_exit(&dnh->dnh_zrlock);
|
|
|
|
|
else
|
|
|
|
|
zrl_remove(&dnh->dnh_zrlock);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static int
|
|
|
|
|
dnode_slots_tryenter(dnode_children_t *children, int idx, int slots)
|
|
|
|
|
{
|
|
|
|
|
ASSERT3S(idx + slots, <=, DNODES_PER_BLOCK);
|
|
|
|
|
|
|
|
|
|
for (int i = idx; i < idx + slots; i++) {
|
|
|
|
|
dnode_handle_t *dnh = &children->dnc_children[i];
|
|
|
|
|
|
|
|
|
|
if (!zrl_tryenter(&dnh->dnh_zrlock)) {
|
|
|
|
|
for (int j = idx; j < i; j++) {
|
|
|
|
|
dnh = &children->dnc_children[j];
|
|
|
|
|
zrl_exit(&dnh->dnh_zrlock);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return (0);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return (1);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void
|
|
|
|
|
dnode_set_slots(dnode_children_t *children, int idx, int slots, void *ptr)
|
|
|
|
|
{
|
|
|
|
|
ASSERT3S(idx + slots, <=, DNODES_PER_BLOCK);
|
|
|
|
|
|
|
|
|
|
for (int i = idx; i < idx + slots; i++) {
|
|
|
|
|
dnode_handle_t *dnh = &children->dnc_children[i];
|
|
|
|
|
dnh->dnh_dnode = ptr;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static boolean_t
|
2018-01-19 12:19:47 +03:00
|
|
|
dnode_check_slots_free(dnode_children_t *children, int idx, int slots)
|
2017-09-06 02:15:04 +03:00
|
|
|
{
|
|
|
|
|
ASSERT3S(idx + slots, <=, DNODES_PER_BLOCK);
|
|
|
|
|
|
2018-04-10 21:15:05 +03:00
|
|
|
/*
|
|
|
|
|
* If all dnode slots are either already free or
|
|
|
|
|
* evictable return B_TRUE.
|
|
|
|
|
*/
|
2017-09-06 02:15:04 +03:00
|
|
|
for (int i = idx; i < idx + slots; i++) {
|
|
|
|
|
dnode_handle_t *dnh = &children->dnc_children[i];
|
2018-01-19 12:19:47 +03:00
|
|
|
dnode_t *dn = dnh->dnh_dnode;
|
|
|
|
|
|
|
|
|
|
if (dn == DN_SLOT_FREE) {
|
|
|
|
|
continue;
|
|
|
|
|
} else if (DN_SLOT_IS_PTR(dn)) {
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
2018-04-10 21:15:05 +03:00
|
|
|
boolean_t can_free = (dn->dn_type == DMU_OT_NONE &&
|
2019-01-11 01:36:23 +03:00
|
|
|
zfs_refcount_is_zero(&dn->dn_holds) &&
|
2018-04-10 21:15:05 +03:00
|
|
|
!DNODE_IS_DIRTY(dn));
|
2018-01-19 12:19:47 +03:00
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
|
2018-04-10 21:15:05 +03:00
|
|
|
if (!can_free)
|
2018-01-19 12:19:47 +03:00
|
|
|
return (B_FALSE);
|
2018-04-10 21:15:05 +03:00
|
|
|
else
|
|
|
|
|
continue;
|
2018-01-19 12:19:47 +03:00
|
|
|
} else {
|
2017-09-06 02:15:04 +03:00
|
|
|
return (B_FALSE);
|
2018-01-19 12:19:47 +03:00
|
|
|
}
|
2017-09-06 02:15:04 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return (B_TRUE);
|
|
|
|
|
}
|
|
|
|
|
|
2018-01-19 12:19:47 +03:00
|
|
|
static void
|
|
|
|
|
dnode_reclaim_slots(dnode_children_t *children, int idx, int slots)
|
|
|
|
|
{
|
|
|
|
|
ASSERT3S(idx + slots, <=, DNODES_PER_BLOCK);
|
|
|
|
|
|
|
|
|
|
for (int i = idx; i < idx + slots; i++) {
|
|
|
|
|
dnode_handle_t *dnh = &children->dnc_children[i];
|
|
|
|
|
|
|
|
|
|
ASSERT(zrl_is_locked(&dnh->dnh_zrlock));
|
|
|
|
|
|
|
|
|
|
if (DN_SLOT_IS_PTR(dnh->dnh_dnode)) {
|
|
|
|
|
ASSERT3S(dnh->dnh_dnode->dn_type, ==, DMU_OT_NONE);
|
|
|
|
|
dnode_destroy(dnh->dnh_dnode);
|
|
|
|
|
dnh->dnh_dnode = DN_SLOT_FREE;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
dnode_free_interior_slots(dnode_t *dn)
|
|
|
|
|
{
|
|
|
|
|
dnode_children_t *children = dmu_buf_get_user(&dn->dn_dbuf->db);
|
|
|
|
|
int epb = dn->dn_dbuf->db.db_size >> DNODE_SHIFT;
|
|
|
|
|
int idx = (dn->dn_object & (epb - 1)) + 1;
|
|
|
|
|
int slots = dn->dn_num_slots - 1;
|
|
|
|
|
|
|
|
|
|
if (slots == 0)
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
ASSERT3S(idx + slots, <=, DNODES_PER_BLOCK);
|
|
|
|
|
|
2019-02-22 20:48:37 +03:00
|
|
|
while (!dnode_slots_tryenter(children, idx, slots)) {
|
2018-01-19 12:19:47 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_free_interior_lock_retry);
|
Cleanup: Use OpenSolaris functions to call scheduler
In our codebase, `cond_resched() and `schedule()` are Linux kernel
functions that have replaced the OpenSolaris `kpreempt()` functions in
the codebase to such an extent that `kpreempt()` in zfs_context.h was
broken. Nobody noticed because we did not actually use it. The header
had defined `kpreempt()` as `yield()`, which works on OpenSolaris and
Illumos where `sched_yield()` is a wrapper for `yield()`, but that does
not work on any other platform.
The FreeBSD platform specific code implemented shims for these, but the
shim for `schedule()` forced us to wait, which is different than merely
rescheduling to another thread as the original Linux code does, while
the shim for `cond_resched()` had the same definition as its kernel
kpreempt() shim.
After studying this, I have concluded that we should reintroduce the
kpreempt() function in platform independent code with the following
definitions:
- In the Linux kernel:
kpreempt(unused) -> cond_resched()
- In the FreeBSD kernel:
kpreempt(unused) -> kern_yield(PRI_USER)
- In userspace:
kpreempt(unused) -> sched_yield()
In userspace, nothing changes from this cleanup. In the kernels, the
function `fm_fini()` will now call `kern_yield(PRI_USER)` on FreeBSD and
`cond_resched()` on Linux. This is instead of `pause("schedule", 1)` on
FreeBSD and `schedule()` on Linux. This makes our behavior consistent
across platforms.
Note that Linux's SPL continues to use `cond_resched()` and
`schedule()`. However, those functions have been removed from both the
FreeBSD code and userspace code.
This should have the benefit of making it slightly easier to port the
code to new platforms by making how things should be mapped less
confusing.
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Neal Gompa <ngompa@datto.com>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes #13845
2022-09-12 19:55:37 +03:00
|
|
|
kpreempt(KPREEMPT_SYNC);
|
2019-02-22 20:48:37 +03:00
|
|
|
}
|
2018-01-19 12:19:47 +03:00
|
|
|
|
|
|
|
|
dnode_set_slots(children, idx, slots, DN_SLOT_FREE);
|
|
|
|
|
dnode_slots_rele(children, idx, slots);
|
|
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
void
|
2010-08-27 01:24:34 +04:00
|
|
|
dnode_special_close(dnode_handle_t *dnh)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
2010-08-27 01:24:34 +04:00
|
|
|
dnode_t *dn = dnh->dnh_dnode;
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
/*
|
2020-03-12 20:25:56 +03:00
|
|
|
* Ensure dnode_rele_and_unlock() has released dn_mtx, after final
|
|
|
|
|
* zfs_refcount_remove()
|
2008-11-20 23:01:55 +03:00
|
|
|
*/
|
2020-03-12 20:25:56 +03:00
|
|
|
mutex_enter(&dn->dn_mtx);
|
|
|
|
|
if (zfs_refcount_count(&dn->dn_holds) > 0)
|
|
|
|
|
cv_wait(&dn->dn_nodnholds, &dn->dn_mtx);
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
ASSERT3U(zfs_refcount_count(&dn->dn_holds), ==, 0);
|
|
|
|
|
|
2015-04-02 06:44:32 +03:00
|
|
|
ASSERT(dn->dn_dbuf == NULL ||
|
|
|
|
|
dmu_buf_get_user(&dn->dn_dbuf->db) == NULL);
|
2010-08-27 01:24:34 +04:00
|
|
|
zrl_add(&dnh->dnh_zrlock);
|
|
|
|
|
dnode_destroy(dn); /* implicit zrl_remove() */
|
|
|
|
|
zrl_destroy(&dnh->dnh_zrlock);
|
|
|
|
|
dnh->dnh_dnode = NULL;
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
2015-04-02 06:44:32 +03:00
|
|
|
void
|
2010-08-27 01:24:34 +04:00
|
|
|
dnode_special_open(objset_t *os, dnode_phys_t *dnp, uint64_t object,
|
|
|
|
|
dnode_handle_t *dnh)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
2015-04-02 06:44:32 +03:00
|
|
|
dnode_t *dn;
|
|
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
zrl_init(&dnh->dnh_zrlock);
|
2020-09-08 21:36:52 +03:00
|
|
|
VERIFY3U(1, ==, zrl_tryenter(&dnh->dnh_zrlock));
|
2017-09-06 02:15:04 +03:00
|
|
|
|
|
|
|
|
dn = dnode_create(os, dnp, NULL, object, dnh);
|
2008-11-20 23:01:55 +03:00
|
|
|
DNODE_VERIFY(dn);
|
2017-09-06 02:15:04 +03:00
|
|
|
|
|
|
|
|
zrl_exit(&dnh->dnh_zrlock);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static void
|
2017-01-27 01:43:28 +03:00
|
|
|
dnode_buf_evict_async(void *dbu)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
2017-09-06 02:15:04 +03:00
|
|
|
dnode_children_t *dnc = dbu;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_buf_evict);
|
|
|
|
|
|
|
|
|
|
for (int i = 0; i < dnc->dnc_count; i++) {
|
|
|
|
|
dnode_handle_t *dnh = &dnc->dnc_children[i];
|
2010-08-27 01:24:34 +04:00
|
|
|
dnode_t *dn;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
/*
|
|
|
|
|
* The dnode handle lock guards against the dnode moving to
|
|
|
|
|
* another valid address, so there is no need here to guard
|
|
|
|
|
* against changes to or from NULL.
|
|
|
|
|
*/
|
2017-09-06 02:15:04 +03:00
|
|
|
if (!DN_SLOT_IS_PTR(dnh->dnh_dnode)) {
|
2010-08-27 01:24:34 +04:00
|
|
|
zrl_destroy(&dnh->dnh_zrlock);
|
2017-09-06 02:15:04 +03:00
|
|
|
dnh->dnh_dnode = DN_SLOT_UNINIT;
|
2008-11-20 23:01:55 +03:00
|
|
|
continue;
|
2010-08-27 01:24:34 +04:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
zrl_add(&dnh->dnh_zrlock);
|
|
|
|
|
dn = dnh->dnh_dnode;
|
2008-11-20 23:01:55 +03:00
|
|
|
/*
|
|
|
|
|
* If there are holds on this dnode, then there should
|
|
|
|
|
* be holds on the dnode's containing dbuf as well; thus
|
2010-08-27 01:24:34 +04:00
|
|
|
* it wouldn't be eligible for eviction and this function
|
2008-11-20 23:01:55 +03:00
|
|
|
* would not have been called.
|
|
|
|
|
*/
|
2018-10-01 20:42:05 +03:00
|
|
|
ASSERT(zfs_refcount_is_zero(&dn->dn_holds));
|
|
|
|
|
ASSERT(zfs_refcount_is_zero(&dn->dn_tx_holds));
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
dnode_destroy(dn); /* implicit zrl_remove() for first slot */
|
2010-08-27 01:24:34 +04:00
|
|
|
zrl_destroy(&dnh->dnh_zrlock);
|
2017-09-06 02:15:04 +03:00
|
|
|
dnh->dnh_dnode = DN_SLOT_UNINIT;
|
2017-08-08 18:38:53 +03:00
|
|
|
}
|
2017-09-06 02:15:04 +03:00
|
|
|
kmem_free(dnc, sizeof (dnode_children_t) +
|
|
|
|
|
dnc->dnc_count * sizeof (dnode_handle_t));
|
2017-08-08 18:38:53 +03:00
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
/*
|
2017-08-29 19:00:28 +03:00
|
|
|
* When the DNODE_MUST_BE_FREE flag is set, the "slots" parameter is used
|
|
|
|
|
* to ensure the hole at the specified object offset is large enough to
|
|
|
|
|
* hold the dnode being created. The slots parameter is also used to ensure
|
|
|
|
|
* a dnode does not span multiple dnode blocks. In both of these cases, if
|
|
|
|
|
* a failure occurs, ENOSPC is returned. Keep in mind, these failure cases
|
|
|
|
|
* are only possible when using DNODE_MUST_BE_FREE.
|
|
|
|
|
*
|
|
|
|
|
* If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
|
|
|
|
|
* dnode_hold_impl() will check if the requested dnode is already consumed
|
|
|
|
|
* as an extra dnode slot by an large dnode, in which case it returns
|
|
|
|
|
* ENOENT.
|
|
|
|
|
*
|
2019-08-28 20:42:02 +03:00
|
|
|
* If the DNODE_DRY_RUN flag is set, we don't actually hold the dnode, just
|
|
|
|
|
* return whether the hold would succeed or not. tag and dnp should set to
|
|
|
|
|
* NULL in this case.
|
|
|
|
|
*
|
2008-11-20 23:01:55 +03:00
|
|
|
* errors:
|
2017-09-06 02:15:04 +03:00
|
|
|
* EINVAL - Invalid object number or flags.
|
|
|
|
|
* ENOSPC - Hole too small to fulfill "slots" request (DNODE_MUST_BE_FREE)
|
|
|
|
|
* EEXIST - Refers to an allocated dnode (DNODE_MUST_BE_FREE)
|
2018-12-05 20:29:33 +03:00
|
|
|
* - Refers to a freeing dnode (DNODE_MUST_BE_FREE)
|
2017-09-06 02:15:04 +03:00
|
|
|
* - Refers to an interior dnode slot (DNODE_MUST_BE_ALLOCATED)
|
|
|
|
|
* ENOENT - The requested dnode is not allocated (DNODE_MUST_BE_ALLOCATED)
|
2018-12-05 20:29:33 +03:00
|
|
|
* - The requested dnode is being freed (DNODE_MUST_BE_ALLOCATED)
|
2017-09-06 02:15:04 +03:00
|
|
|
* EIO - I/O error when reading the meta dnode dbuf.
|
|
|
|
|
*
|
2008-11-20 23:01:55 +03:00
|
|
|
* succeeds even for free dnodes.
|
|
|
|
|
*/
|
|
|
|
|
int
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
dnode_hold_impl(objset_t *os, uint64_t object, int flag, int slots,
|
2022-04-19 21:38:30 +03:00
|
|
|
const void *tag, dnode_t **dnp)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
2017-09-06 02:15:04 +03:00
|
|
|
int epb, idx, err;
|
2008-11-20 23:01:55 +03:00
|
|
|
int drop_struct_lock = FALSE;
|
|
|
|
|
int type;
|
|
|
|
|
uint64_t blk;
|
|
|
|
|
dnode_t *mdn, *dn;
|
|
|
|
|
dmu_buf_impl_t *db;
|
2017-09-06 02:15:04 +03:00
|
|
|
dnode_children_t *dnc;
|
|
|
|
|
dnode_phys_t *dn_block;
|
2010-08-27 01:24:34 +04:00
|
|
|
dnode_handle_t *dnh;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
ASSERT(!(flag & DNODE_MUST_BE_ALLOCATED) || (slots == 0));
|
|
|
|
|
ASSERT(!(flag & DNODE_MUST_BE_FREE) || (slots > 0));
|
2019-08-28 20:42:02 +03:00
|
|
|
IMPLY(flag & DNODE_DRY_RUN, (tag == NULL) && (dnp == NULL));
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
/*
|
|
|
|
|
* If you are holding the spa config lock as writer, you shouldn't
|
2010-05-29 00:45:14 +04:00
|
|
|
* be asking the DMU to do *anything* unless it's the root pool
|
|
|
|
|
* which may require us to read from the root filesystem while
|
|
|
|
|
* holding some (not all) of the locks as writer.
|
2008-12-03 23:09:06 +03:00
|
|
|
*/
|
2010-05-29 00:45:14 +04:00
|
|
|
ASSERT(spa_config_held(os->os_spa, SCL_ALL, RW_WRITER) == 0 ||
|
|
|
|
|
(spa_is_root(os->os_spa) &&
|
2010-08-27 01:24:34 +04:00
|
|
|
spa_config_held(os->os_spa, SCL_STATE, RW_WRITER)));
|
2008-12-03 23:09:06 +03:00
|
|
|
|
2016-12-17 01:11:29 +03:00
|
|
|
ASSERT((flag & DNODE_MUST_BE_ALLOCATED) || (flag & DNODE_MUST_BE_FREE));
|
|
|
|
|
|
2018-02-14 01:54:54 +03:00
|
|
|
if (object == DMU_USERUSED_OBJECT || object == DMU_GROUPUSED_OBJECT ||
|
|
|
|
|
object == DMU_PROJECTUSED_OBJECT) {
|
|
|
|
|
if (object == DMU_USERUSED_OBJECT)
|
|
|
|
|
dn = DMU_USERUSED_DNODE(os);
|
|
|
|
|
else if (object == DMU_GROUPUSED_OBJECT)
|
|
|
|
|
dn = DMU_GROUPUSED_DNODE(os);
|
|
|
|
|
else
|
|
|
|
|
dn = DMU_PROJECTUSED_DNODE(os);
|
2009-07-03 02:44:48 +04:00
|
|
|
if (dn == NULL)
|
2013-03-08 22:41:28 +04:00
|
|
|
return (SET_ERROR(ENOENT));
|
2009-07-03 02:44:48 +04:00
|
|
|
type = dn->dn_type;
|
|
|
|
|
if ((flag & DNODE_MUST_BE_ALLOCATED) && type == DMU_OT_NONE)
|
2013-03-08 22:41:28 +04:00
|
|
|
return (SET_ERROR(ENOENT));
|
2009-07-03 02:44:48 +04:00
|
|
|
if ((flag & DNODE_MUST_BE_FREE) && type != DMU_OT_NONE)
|
2013-03-08 22:41:28 +04:00
|
|
|
return (SET_ERROR(EEXIST));
|
2009-07-03 02:44:48 +04:00
|
|
|
DNODE_VERIFY(dn);
|
2019-08-28 20:42:02 +03:00
|
|
|
/* Don't actually hold if dry run, just return 0 */
|
|
|
|
|
if (!(flag & DNODE_DRY_RUN)) {
|
|
|
|
|
(void) zfs_refcount_add(&dn->dn_holds, tag);
|
|
|
|
|
*dnp = dn;
|
|
|
|
|
}
|
2009-07-03 02:44:48 +04:00
|
|
|
return (0);
|
|
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
if (object == 0 || object >= DN_MAX_OBJECT)
|
2013-03-08 22:41:28 +04:00
|
|
|
return (SET_ERROR(EINVAL));
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
mdn = DMU_META_DNODE(os);
|
|
|
|
|
ASSERT(mdn->dn_object == DMU_META_DNODE_OBJECT);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
DNODE_VERIFY(mdn);
|
|
|
|
|
|
|
|
|
|
if (!RW_WRITE_HELD(&mdn->dn_struct_rwlock)) {
|
|
|
|
|
rw_enter(&mdn->dn_struct_rwlock, RW_READER);
|
|
|
|
|
drop_struct_lock = TRUE;
|
|
|
|
|
}
|
|
|
|
|
|
2015-12-22 04:31:57 +03:00
|
|
|
blk = dbuf_whichblock(mdn, 0, object * sizeof (dnode_phys_t));
|
2008-11-20 23:01:55 +03:00
|
|
|
db = dbuf_hold(mdn, blk, FTAG);
|
|
|
|
|
if (drop_struct_lock)
|
|
|
|
|
rw_exit(&mdn->dn_struct_rwlock);
|
2017-09-06 02:15:04 +03:00
|
|
|
if (db == NULL) {
|
|
|
|
|
DNODE_STAT_BUMP(dnode_hold_dbuf_hold);
|
2013-03-08 22:41:28 +04:00
|
|
|
return (SET_ERROR(EIO));
|
2017-09-06 02:15:04 +03:00
|
|
|
}
|
Native Encryption for ZFS on Linux
This change incorporates three major pieces:
The first change is a keystore that manages wrapping
and encryption keys for encrypted datasets. These
commands mostly involve manipulating the new
DSL Crypto Key ZAP Objects that live in the MOS. Each
encrypted dataset has its own DSL Crypto Key that is
protected with a user's key. This level of indirection
allows users to change their keys without re-encrypting
their entire datasets. The change implements the new
subcommands "zfs load-key", "zfs unload-key" and
"zfs change-key" which allow the user to manage their
encryption keys and settings. In addition, several new
flags and properties have been added to allow dataset
creation and to make mounting and unmounting more
convenient.
The second piece of this patch provides the ability to
encrypt, decyrpt, and authenticate protected datasets.
Each object set maintains a Merkel tree of Message
Authentication Codes that protect the lower layers,
similarly to how checksums are maintained. This part
impacts the zio layer, which handles the actual
encryption and generation of MACs, as well as the ARC
and DMU, which need to be able to handle encrypted
buffers and protected data.
The last addition is the ability to do raw, encrypted
sends and receives. The idea here is to send raw
encrypted and compressed data and receive it exactly
as is on a backup system. This means that the dataset
on the receiving system is protected using the same
user key that is in use on the sending side. By doing
so, datasets can be efficiently backed up to an
untrusted system without fear of data being
compromised.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #494
Closes #5769
2017-08-14 20:36:48 +03:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We do not need to decrypt to read the dnode so it doesn't matter
|
|
|
|
|
* if we get the encrypted or decrypted version.
|
|
|
|
|
*/
|
2020-09-25 23:49:22 +03:00
|
|
|
err = dbuf_read(db, NULL, DB_RF_CANFAIL |
|
|
|
|
|
DB_RF_NO_DECRYPT | DB_RF_NOPREFETCH);
|
2008-11-20 23:01:55 +03:00
|
|
|
if (err) {
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_hold_dbuf_read);
|
2008-11-20 23:01:55 +03:00
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (err);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
ASSERT3U(db->db.db_size, >=, 1<<DNODE_SHIFT);
|
|
|
|
|
epb = db->db.db_size >> DNODE_SHIFT;
|
|
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
idx = object & (epb - 1);
|
|
|
|
|
dn_block = (dnode_phys_t *)db->db.db_data;
|
|
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
ASSERT(DB_DNODE(db)->dn_type == DMU_OT_DNODE);
|
2017-09-06 02:15:04 +03:00
|
|
|
dnc = dmu_buf_get_user(&db->db);
|
|
|
|
|
dnh = NULL;
|
|
|
|
|
if (dnc == NULL) {
|
2010-08-27 01:24:34 +04:00
|
|
|
dnode_children_t *winner;
|
2017-09-06 02:15:04 +03:00
|
|
|
int skip = 0;
|
|
|
|
|
|
|
|
|
|
dnc = kmem_zalloc(sizeof (dnode_children_t) +
|
2015-04-01 18:09:20 +03:00
|
|
|
epb * sizeof (dnode_handle_t), KM_SLEEP);
|
2017-09-06 02:15:04 +03:00
|
|
|
dnc->dnc_count = epb;
|
|
|
|
|
dnh = &dnc->dnc_children[0];
|
|
|
|
|
|
|
|
|
|
/* Initialize dnode slot status from dnode_phys_t */
|
|
|
|
|
for (int i = 0; i < epb; i++) {
|
2010-08-27 01:24:34 +04:00
|
|
|
zrl_init(&dnh[i].dnh_zrlock);
|
2017-09-06 02:15:04 +03:00
|
|
|
|
|
|
|
|
if (skip) {
|
|
|
|
|
skip--;
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (dn_block[i].dn_type != DMU_OT_NONE) {
|
|
|
|
|
int interior = dn_block[i].dn_extra_slots;
|
|
|
|
|
|
|
|
|
|
dnode_set_slots(dnc, i, 1, DN_SLOT_ALLOCATED);
|
|
|
|
|
dnode_set_slots(dnc, i + 1, interior,
|
|
|
|
|
DN_SLOT_INTERIOR);
|
|
|
|
|
skip = interior;
|
|
|
|
|
} else {
|
|
|
|
|
dnh[i].dnh_dnode = DN_SLOT_FREE;
|
|
|
|
|
skip = 0;
|
|
|
|
|
}
|
2010-08-27 01:24:34 +04:00
|
|
|
}
|
2017-09-06 02:15:04 +03:00
|
|
|
|
|
|
|
|
dmu_buf_init_user(&dnc->dnc_dbu, NULL,
|
2017-01-27 01:43:28 +03:00
|
|
|
dnode_buf_evict_async, NULL);
|
2017-09-06 02:15:04 +03:00
|
|
|
winner = dmu_buf_set_user(&db->db, &dnc->dnc_dbu);
|
2015-04-02 06:44:32 +03:00
|
|
|
if (winner != NULL) {
|
2015-04-01 16:49:14 +03:00
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
for (int i = 0; i < epb; i++)
|
2015-04-01 16:49:14 +03:00
|
|
|
zrl_destroy(&dnh[i].dnh_zrlock);
|
|
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
kmem_free(dnc, sizeof (dnode_children_t) +
|
2015-04-01 18:09:20 +03:00
|
|
|
epb * sizeof (dnode_handle_t));
|
2017-09-06 02:15:04 +03:00
|
|
|
dnc = winner;
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
ASSERT(dnc->dnc_count == epb);
|
|
|
|
|
|
|
|
|
|
if (flag & DNODE_MUST_BE_ALLOCATED) {
|
|
|
|
|
slots = 1;
|
|
|
|
|
|
2019-02-22 20:48:37 +03:00
|
|
|
dnode_slots_hold(dnc, idx, slots);
|
|
|
|
|
dnh = &dnc->dnc_children[idx];
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
|
2019-02-22 20:48:37 +03:00
|
|
|
if (DN_SLOT_IS_PTR(dnh->dnh_dnode)) {
|
|
|
|
|
dn = dnh->dnh_dnode;
|
|
|
|
|
} else if (dnh->dnh_dnode == DN_SLOT_INTERIOR) {
|
|
|
|
|
DNODE_STAT_BUMP(dnode_hold_alloc_interior);
|
2017-09-06 02:15:04 +03:00
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
2019-02-22 20:48:37 +03:00
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (SET_ERROR(EEXIST));
|
|
|
|
|
} else if (dnh->dnh_dnode != DN_SLOT_ALLOCATED) {
|
|
|
|
|
DNODE_STAT_BUMP(dnode_hold_alloc_misses);
|
|
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (SET_ERROR(ENOENT));
|
|
|
|
|
} else {
|
|
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
|
|
|
|
while (!dnode_slots_tryenter(dnc, idx, slots)) {
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_hold_alloc_lock_retry);
|
Cleanup: Use OpenSolaris functions to call scheduler
In our codebase, `cond_resched() and `schedule()` are Linux kernel
functions that have replaced the OpenSolaris `kpreempt()` functions in
the codebase to such an extent that `kpreempt()` in zfs_context.h was
broken. Nobody noticed because we did not actually use it. The header
had defined `kpreempt()` as `yield()`, which works on OpenSolaris and
Illumos where `sched_yield()` is a wrapper for `yield()`, but that does
not work on any other platform.
The FreeBSD platform specific code implemented shims for these, but the
shim for `schedule()` forced us to wait, which is different than merely
rescheduling to another thread as the original Linux code does, while
the shim for `cond_resched()` had the same definition as its kernel
kpreempt() shim.
After studying this, I have concluded that we should reintroduce the
kpreempt() function in platform independent code with the following
definitions:
- In the Linux kernel:
kpreempt(unused) -> cond_resched()
- In the FreeBSD kernel:
kpreempt(unused) -> kern_yield(PRI_USER)
- In userspace:
kpreempt(unused) -> sched_yield()
In userspace, nothing changes from this cleanup. In the kernels, the
function `fm_fini()` will now call `kern_yield(PRI_USER)` on FreeBSD and
`cond_resched()` on Linux. This is instead of `pause("schedule", 1)` on
FreeBSD and `schedule()` on Linux. This makes our behavior consistent
across platforms.
Note that Linux's SPL continues to use `cond_resched()` and
`schedule()`. However, those functions have been removed from both the
FreeBSD code and userspace code.
This should have the benefit of making it slightly easier to port the
code to new platforms by making how things should be mapped less
confusing.
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Neal Gompa <ngompa@datto.com>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes #13845
2022-09-12 19:55:37 +03:00
|
|
|
kpreempt(KPREEMPT_SYNC);
|
2017-09-06 02:15:04 +03:00
|
|
|
}
|
2017-08-08 18:38:53 +03:00
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
/*
|
|
|
|
|
* Someone else won the race and called dnode_create()
|
|
|
|
|
* after we checked DN_SLOT_IS_PTR() above but before
|
|
|
|
|
* we acquired the lock.
|
|
|
|
|
*/
|
|
|
|
|
if (DN_SLOT_IS_PTR(dnh->dnh_dnode)) {
|
|
|
|
|
DNODE_STAT_BUMP(dnode_hold_alloc_lock_misses);
|
|
|
|
|
dn = dnh->dnh_dnode;
|
|
|
|
|
} else {
|
|
|
|
|
dn = dnode_create(os, dn_block + idx, db,
|
|
|
|
|
object, dnh);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
2018-12-05 20:29:33 +03:00
|
|
|
if (dn->dn_type == DMU_OT_NONE || dn->dn_free_txg != 0) {
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_hold_alloc_type_none);
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (SET_ERROR(ENOENT));
|
|
|
|
|
}
|
|
|
|
|
|
2019-08-28 20:42:02 +03:00
|
|
|
/* Don't actually hold if dry run, just return 0 */
|
|
|
|
|
if (flag & DNODE_DRY_RUN) {
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (0);
|
|
|
|
|
}
|
|
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_hold_alloc_hits);
|
|
|
|
|
} else if (flag & DNODE_MUST_BE_FREE) {
|
|
|
|
|
|
|
|
|
|
if (idx + slots - 1 >= DNODES_PER_BLOCK) {
|
|
|
|
|
DNODE_STAT_BUMP(dnode_hold_free_overflow);
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (SET_ERROR(ENOSPC));
|
|
|
|
|
}
|
|
|
|
|
|
2019-02-22 20:48:37 +03:00
|
|
|
dnode_slots_hold(dnc, idx, slots);
|
2017-09-06 02:15:04 +03:00
|
|
|
|
2019-02-22 20:48:37 +03:00
|
|
|
if (!dnode_check_slots_free(dnc, idx, slots)) {
|
|
|
|
|
DNODE_STAT_BUMP(dnode_hold_free_misses);
|
2017-09-06 02:15:04 +03:00
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
2019-02-22 20:48:37 +03:00
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (SET_ERROR(ENOSPC));
|
|
|
|
|
}
|
2017-09-06 02:15:04 +03:00
|
|
|
|
2019-02-22 20:48:37 +03:00
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
|
|
|
|
while (!dnode_slots_tryenter(dnc, idx, slots)) {
|
|
|
|
|
DNODE_STAT_BUMP(dnode_hold_free_lock_retry);
|
Cleanup: Use OpenSolaris functions to call scheduler
In our codebase, `cond_resched() and `schedule()` are Linux kernel
functions that have replaced the OpenSolaris `kpreempt()` functions in
the codebase to such an extent that `kpreempt()` in zfs_context.h was
broken. Nobody noticed because we did not actually use it. The header
had defined `kpreempt()` as `yield()`, which works on OpenSolaris and
Illumos where `sched_yield()` is a wrapper for `yield()`, but that does
not work on any other platform.
The FreeBSD platform specific code implemented shims for these, but the
shim for `schedule()` forced us to wait, which is different than merely
rescheduling to another thread as the original Linux code does, while
the shim for `cond_resched()` had the same definition as its kernel
kpreempt() shim.
After studying this, I have concluded that we should reintroduce the
kpreempt() function in platform independent code with the following
definitions:
- In the Linux kernel:
kpreempt(unused) -> cond_resched()
- In the FreeBSD kernel:
kpreempt(unused) -> kern_yield(PRI_USER)
- In userspace:
kpreempt(unused) -> sched_yield()
In userspace, nothing changes from this cleanup. In the kernels, the
function `fm_fini()` will now call `kern_yield(PRI_USER)` on FreeBSD and
`cond_resched()` on Linux. This is instead of `pause("schedule", 1)` on
FreeBSD and `schedule()` on Linux. This makes our behavior consistent
across platforms.
Note that Linux's SPL continues to use `cond_resched()` and
`schedule()`. However, those functions have been removed from both the
FreeBSD code and userspace code.
This should have the benefit of making it slightly easier to port the
code to new platforms by making how things should be mapped less
confusing.
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Neal Gompa <ngompa@datto.com>
Signed-off-by: Richard Yao <richard.yao@alumni.stonybrook.edu>
Closes #13845
2022-09-12 19:55:37 +03:00
|
|
|
kpreempt(KPREEMPT_SYNC);
|
2019-02-22 20:48:37 +03:00
|
|
|
}
|
2017-09-06 02:15:04 +03:00
|
|
|
|
2019-02-22 20:48:37 +03:00
|
|
|
if (!dnode_check_slots_free(dnc, idx, slots)) {
|
|
|
|
|
DNODE_STAT_BUMP(dnode_hold_free_lock_misses);
|
|
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (SET_ERROR(ENOSPC));
|
|
|
|
|
}
|
2018-01-19 12:19:47 +03:00
|
|
|
|
2019-02-22 20:48:37 +03:00
|
|
|
/*
|
|
|
|
|
* Allocated but otherwise free dnodes which would
|
|
|
|
|
* be in the interior of a multi-slot dnodes need
|
|
|
|
|
* to be freed. Single slot dnodes can be safely
|
|
|
|
|
* re-purposed as a performance optimization.
|
|
|
|
|
*/
|
|
|
|
|
if (slots > 1)
|
|
|
|
|
dnode_reclaim_slots(dnc, idx + 1, slots - 1);
|
|
|
|
|
|
|
|
|
|
dnh = &dnc->dnc_children[idx];
|
|
|
|
|
if (DN_SLOT_IS_PTR(dnh->dnh_dnode)) {
|
|
|
|
|
dn = dnh->dnh_dnode;
|
|
|
|
|
} else {
|
|
|
|
|
dn = dnode_create(os, dn_block + idx, db,
|
|
|
|
|
object, dnh);
|
2017-09-06 02:15:04 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
2018-12-05 20:29:33 +03:00
|
|
|
if (!zfs_refcount_is_zero(&dn->dn_holds) || dn->dn_free_txg) {
|
2017-09-06 02:15:04 +03:00
|
|
|
DNODE_STAT_BUMP(dnode_hold_free_refcount);
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (SET_ERROR(EEXIST));
|
|
|
|
|
}
|
|
|
|
|
|
2019-08-28 20:42:02 +03:00
|
|
|
/* Don't actually hold if dry run, just return 0 */
|
|
|
|
|
if (flag & DNODE_DRY_RUN) {
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (0);
|
|
|
|
|
}
|
|
|
|
|
|
2017-09-06 02:15:04 +03:00
|
|
|
dnode_set_slots(dnc, idx + 1, slots - 1, DN_SLOT_INTERIOR);
|
|
|
|
|
DNODE_STAT_BUMP(dnode_hold_free_hits);
|
|
|
|
|
} else {
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
dbuf_rele(db, FTAG);
|
2017-09-06 02:15:04 +03:00
|
|
|
return (SET_ERROR(EINVAL));
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
}
|
|
|
|
|
|
2019-08-28 20:42:02 +03:00
|
|
|
ASSERT0(dn->dn_free_txg);
|
2017-09-06 02:15:04 +03:00
|
|
|
|
2018-09-26 20:29:26 +03:00
|
|
|
if (zfs_refcount_add(&dn->dn_holds, tag) == 1)
|
2010-08-27 01:24:34 +04:00
|
|
|
dbuf_add_ref(db, dnh);
|
2017-08-08 18:38:53 +03:00
|
|
|
|
2015-04-02 06:44:32 +03:00
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
|
2010-08-27 01:24:34 +04:00
|
|
|
/* Now we can rely on the hold to prevent the dnode from moving. */
|
2017-09-06 02:15:04 +03:00
|
|
|
dnode_slots_rele(dnc, idx, slots);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
DNODE_VERIFY(dn);
|
2019-12-16 21:55:11 +03:00
|
|
|
ASSERT3P(dnp, !=, NULL);
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT3P(dn->dn_dbuf, ==, db);
|
|
|
|
|
ASSERT3U(dn->dn_object, ==, object);
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
|
|
|
|
|
*dnp = dn;
|
|
|
|
|
return (0);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Return held dnode if the object is allocated, NULL if not.
|
|
|
|
|
*/
|
|
|
|
|
int
|
2022-04-19 21:38:30 +03:00
|
|
|
dnode_hold(objset_t *os, uint64_t object, const void *tag, dnode_t **dnp)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
return (dnode_hold_impl(os, object, DNODE_MUST_BE_ALLOCATED, 0, tag,
|
|
|
|
|
dnp));
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Can only add a reference if there is already at least one
|
|
|
|
|
* reference on the dnode. Returns FALSE if unable to add a
|
|
|
|
|
* new reference.
|
|
|
|
|
*/
|
|
|
|
|
boolean_t
|
2022-04-19 21:38:30 +03:00
|
|
|
dnode_add_ref(dnode_t *dn, const void *tag)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
2018-10-01 20:42:05 +03:00
|
|
|
if (zfs_refcount_is_zero(&dn->dn_holds)) {
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
return (FALSE);
|
|
|
|
|
}
|
2018-09-26 20:29:26 +03:00
|
|
|
VERIFY(1 < zfs_refcount_add(&dn->dn_holds, tag));
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
return (TRUE);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
2022-04-19 21:38:30 +03:00
|
|
|
dnode_rele(dnode_t *dn, const void *tag)
|
2015-03-12 03:10:35 +03:00
|
|
|
{
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
2018-08-01 00:51:15 +03:00
|
|
|
dnode_rele_and_unlock(dn, tag, B_FALSE);
|
2015-03-12 03:10:35 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
2022-04-19 21:38:30 +03:00
|
|
|
dnode_rele_and_unlock(dnode_t *dn, const void *tag, boolean_t evicting)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
|
|
|
|
uint64_t refs;
|
2010-08-27 01:24:34 +04:00
|
|
|
/* Get while the hold prevents the dnode from moving. */
|
|
|
|
|
dmu_buf_impl_t *db = dn->dn_dbuf;
|
|
|
|
|
dnode_handle_t *dnh = dn->dn_handle;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2018-10-01 20:42:05 +03:00
|
|
|
refs = zfs_refcount_remove(&dn->dn_holds, tag);
|
2020-03-12 20:25:56 +03:00
|
|
|
if (refs == 0)
|
|
|
|
|
cv_broadcast(&dn->dn_nodnholds);
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_exit(&dn->dn_mtx);
|
2020-03-12 20:25:56 +03:00
|
|
|
/* dnode could get destroyed at this point, so don't use it anymore */
|
2010-08-27 01:24:34 +04:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* It's unsafe to release the last hold on a dnode by dnode_rele() or
|
|
|
|
|
* indirectly by dbuf_rele() while relying on the dnode handle to
|
|
|
|
|
* prevent the dnode from moving, since releasing the last hold could
|
|
|
|
|
* result in the dnode's parent dbuf evicting its dnode handles. For
|
|
|
|
|
* that reason anyone calling dnode_rele() or dbuf_rele() without some
|
|
|
|
|
* other direct or indirect hold on the dnode must first drop the dnode
|
|
|
|
|
* handle.
|
|
|
|
|
*/
|
2021-12-21 20:38:30 +03:00
|
|
|
#ifdef ZFS_DEBUG
|
2010-08-27 01:24:34 +04:00
|
|
|
ASSERT(refs > 0 || dnh->dnh_zrlock.zr_owner != curthread);
|
2021-12-21 20:38:30 +03:00
|
|
|
#endif
|
2010-08-27 01:24:34 +04:00
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
/* NOTE: the DNODE_DNODE does not have a dn_dbuf */
|
2010-08-27 01:24:34 +04:00
|
|
|
if (refs == 0 && db != NULL) {
|
|
|
|
|
/*
|
|
|
|
|
* Another thread could add a hold to the dnode handle in
|
|
|
|
|
* dnode_hold_impl() while holding the parent dbuf. Since the
|
|
|
|
|
* hold on the parent dbuf prevents the handle from being
|
|
|
|
|
* destroyed, the hold on the handle is OK. We can't yet assert
|
|
|
|
|
* that the handle has zero references, but that will be
|
|
|
|
|
* asserted anyway when the handle gets destroyed.
|
|
|
|
|
*/
|
2018-05-31 20:29:12 +03:00
|
|
|
mutex_enter(&db->db_mtx);
|
2018-08-01 00:51:15 +03:00
|
|
|
dbuf_rele_and_unlock(db, dnh, evicting);
|
2010-08-27 01:24:34 +04:00
|
|
|
}
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
2019-08-28 20:42:02 +03:00
|
|
|
/*
|
|
|
|
|
* Test whether we can create a dnode at the specified location.
|
|
|
|
|
*/
|
|
|
|
|
int
|
|
|
|
|
dnode_try_claim(objset_t *os, uint64_t object, int slots)
|
|
|
|
|
{
|
|
|
|
|
return (dnode_hold_impl(os, object, DNODE_MUST_BE_FREE | DNODE_DRY_RUN,
|
|
|
|
|
slots, NULL, NULL));
|
|
|
|
|
}
|
|
|
|
|
|
2021-11-08 00:27:44 +03:00
|
|
|
/*
|
dnode_is_dirty: check dnode and its data for dirtiness
Over its history this the dirty dnode test has been changed between
checking for a dnodes being on `os_dirty_dnodes` (`dn_dirty_link`) and
`dn_dirty_record`.
de198f2d9 Fix lseek(SEEK_DATA/SEEK_HOLE) mmap consistency
2531ce372 Revert "Report holes when there are only metadata changes"
ec4f9b8f3 Report holes when there are only metadata changes
454365bba Fix dirty check in dmu_offset_next()
66aca2473 SEEK_HOLE should not block on txg_wait_synced()
Also illumos/illumos-gate@c543ec060d illumos/illumos-gate@2bcf0248e9
It turns out both are actually required.
In the case of appending data to a newly created file, the dnode proper
is dirtied (at least to change the blocksize) and dirty records are
added. Thus, a single logical operation is represented by separate
dirty indicators, and must not be separated.
The incorrect dirty check becomes a problem when the first block of a
file is being appended to while another process is calling lseek to skip
holes. There is a small window where the dnode part is undirtied while
there are still dirty records. In this case, `lseek(fd, 0, SEEK_DATA)`
would not know that the file is dirty, and would go to
`dnode_next_offset()`. Since the object has no data blocks yet, it
returns `ESRCH`, indicating no data found, which results in `ENXIO`
being returned to `lseek()`'s caller.
Since coreutils 9.2, `cp` performs sparse copies by default, that is, it
uses `SEEK_DATA` and `SEEK_HOLE` against the source file and attempts to
replicate the holes in the target. When it hits the bug, its initial
search for data fails, and it goes on to call `fallocate()` to create a
hole over the entire destination file.
This has come up more recently as users upgrade their systems, getting
OpenZFS 2.2 as well as a newer coreutils. However, this problem has been
reproduced against 2.1, as well as on FreeBSD 13 and 14.
This change simply updates the dirty check to check both types of dirty.
If there's anything dirty at all, we immediately go to the "wait for
sync" stage, It doesn't really matter after that; both changes are on
disk, so the dirty fields should be correct.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
Closes #15571
Closes #15526
2023-11-28 20:15:48 +03:00
|
|
|
* Checks if the dnode itself is dirty, or is carrying any uncommitted records.
|
|
|
|
|
* It is important to check both conditions, as some operations (eg appending
|
|
|
|
|
* to a file) can dirty both as a single logical unit, but they are not synced
|
|
|
|
|
* out atomically, so checking one and not the other can result in an object
|
|
|
|
|
* appearing to be clean mid-way through a commit.
|
|
|
|
|
*
|
|
|
|
|
* Do not change this lightly! If you get it wrong, dmu_offset_next() can
|
|
|
|
|
* detect a hole where there is really data, leading to silent corruption.
|
2021-11-08 00:27:44 +03:00
|
|
|
*/
|
|
|
|
|
boolean_t
|
|
|
|
|
dnode_is_dirty(dnode_t *dn)
|
|
|
|
|
{
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
2023-04-18 18:41:52 +03:00
|
|
|
|
2021-11-08 00:27:44 +03:00
|
|
|
for (int i = 0; i < TXG_SIZE; i++) {
|
dnode_is_dirty: check dnode and its data for dirtiness
Over its history this the dirty dnode test has been changed between
checking for a dnodes being on `os_dirty_dnodes` (`dn_dirty_link`) and
`dn_dirty_record`.
de198f2d9 Fix lseek(SEEK_DATA/SEEK_HOLE) mmap consistency
2531ce372 Revert "Report holes when there are only metadata changes"
ec4f9b8f3 Report holes when there are only metadata changes
454365bba Fix dirty check in dmu_offset_next()
66aca2473 SEEK_HOLE should not block on txg_wait_synced()
Also illumos/illumos-gate@c543ec060d illumos/illumos-gate@2bcf0248e9
It turns out both are actually required.
In the case of appending data to a newly created file, the dnode proper
is dirtied (at least to change the blocksize) and dirty records are
added. Thus, a single logical operation is represented by separate
dirty indicators, and must not be separated.
The incorrect dirty check becomes a problem when the first block of a
file is being appended to while another process is calling lseek to skip
holes. There is a small window where the dnode part is undirtied while
there are still dirty records. In this case, `lseek(fd, 0, SEEK_DATA)`
would not know that the file is dirty, and would go to
`dnode_next_offset()`. Since the object has no data blocks yet, it
returns `ESRCH`, indicating no data found, which results in `ENXIO`
being returned to `lseek()`'s caller.
Since coreutils 9.2, `cp` performs sparse copies by default, that is, it
uses `SEEK_DATA` and `SEEK_HOLE` against the source file and attempts to
replicate the holes in the target. When it hits the bug, its initial
search for data fails, and it goes on to call `fallocate()` to create a
hole over the entire destination file.
This has come up more recently as users upgrade their systems, getting
OpenZFS 2.2 as well as a newer coreutils. However, this problem has been
reproduced against 2.1, as well as on FreeBSD 13 and 14.
This change simply updates the dirty check to check both types of dirty.
If there's anything dirty at all, we immediately go to the "wait for
sync" stage, It doesn't really matter after that; both changes are on
disk, so the dirty fields should be correct.
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Rich Ercolani <rincebrain@gmail.com>
Signed-off-by: Rob Norris <rob.norris@klarasystems.com>
Closes #15571
Closes #15526
2023-11-28 20:15:48 +03:00
|
|
|
if (multilist_link_active(&dn->dn_dirty_link[i]) ||
|
|
|
|
|
!list_is_empty(&dn->dn_dirty_records[i])) {
|
2021-11-08 00:27:44 +03:00
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
return (B_TRUE);
|
|
|
|
|
}
|
|
|
|
|
}
|
2023-04-18 18:41:52 +03:00
|
|
|
|
2021-11-08 00:27:44 +03:00
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
|
|
|
|
|
return (B_FALSE);
|
|
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
void
|
|
|
|
|
dnode_setdirty(dnode_t *dn, dmu_tx_t *tx)
|
|
|
|
|
{
|
2010-05-29 00:45:14 +04:00
|
|
|
objset_t *os = dn->dn_objset;
|
2008-11-20 23:01:55 +03:00
|
|
|
uint64_t txg = tx->tx_txg;
|
|
|
|
|
|
2009-07-03 02:44:48 +04:00
|
|
|
if (DMU_OBJECT_IS_SPECIAL(dn->dn_object)) {
|
|
|
|
|
dsl_dataset_dirty(os->os_dsl_dataset, tx);
|
2008-11-20 23:01:55 +03:00
|
|
|
return;
|
2009-07-03 02:44:48 +04:00
|
|
|
}
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
DNODE_VERIFY(dn);
|
|
|
|
|
|
|
|
|
|
#ifdef ZFS_DEBUG
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
|
|
|
|
ASSERT(dn->dn_phys->dn_type || dn->dn_allocated_txg);
|
2010-08-27 01:24:34 +04:00
|
|
|
ASSERT(dn->dn_free_txg == 0 || dn->dn_free_txg >= txg);
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
#endif
|
|
|
|
|
|
2010-05-29 00:45:14 +04:00
|
|
|
/*
|
|
|
|
|
* Determine old uid/gid when necessary
|
|
|
|
|
*/
|
|
|
|
|
dmu_objset_userquota_get_ids(dn, B_TRUE, tx);
|
|
|
|
|
|
2021-06-10 19:42:31 +03:00
|
|
|
multilist_t *dirtylist = &os->os_dirty_dnodes[txg & TXG_MASK];
|
2017-03-21 04:36:00 +03:00
|
|
|
multilist_sublist_t *mls = multilist_sublist_lock_obj(dirtylist, dn);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If we are already marked dirty, we're done.
|
|
|
|
|
*/
|
2018-04-10 21:15:05 +03:00
|
|
|
if (multilist_link_active(&dn->dn_dirty_link[txg & TXG_MASK])) {
|
2017-03-21 04:36:00 +03:00
|
|
|
multilist_sublist_unlock(mls);
|
2008-11-20 23:01:55 +03:00
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
2018-10-01 20:42:05 +03:00
|
|
|
ASSERT(!zfs_refcount_is_zero(&dn->dn_holds) ||
|
2015-04-03 06:14:28 +03:00
|
|
|
!avl_is_empty(&dn->dn_dbufs));
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT(dn->dn_datablksz != 0);
|
2019-04-12 21:30:59 +03:00
|
|
|
ASSERT0(dn->dn_next_bonuslen[txg & TXG_MASK]);
|
|
|
|
|
ASSERT0(dn->dn_next_blksz[txg & TXG_MASK]);
|
|
|
|
|
ASSERT0(dn->dn_next_bonustype[txg & TXG_MASK]);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
dprintf_ds(os->os_dsl_dataset, "obj=%llu txg=%llu\n",
|
2021-06-23 07:53:45 +03:00
|
|
|
(u_longlong_t)dn->dn_object, (u_longlong_t)txg);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2017-03-21 04:36:00 +03:00
|
|
|
multilist_sublist_insert_head(mls, dn);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2017-03-21 04:36:00 +03:00
|
|
|
multilist_sublist_unlock(mls);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The dnode maintains a hold on its containing dbuf as
|
|
|
|
|
* long as there are holds on it. Each instantiated child
|
2010-08-27 01:24:34 +04:00
|
|
|
* dbuf maintains a hold on the dnode. When the last child
|
2008-11-20 23:01:55 +03:00
|
|
|
* drops its hold, the dnode will drop its hold on the
|
|
|
|
|
* containing dbuf. We add a "dirty hold" here so that the
|
|
|
|
|
* dnode will hang around after we finish processing its
|
|
|
|
|
* children.
|
|
|
|
|
*/
|
|
|
|
|
VERIFY(dnode_add_ref(dn, (void *)(uintptr_t)tx->tx_txg));
|
|
|
|
|
|
|
|
|
|
(void) dbuf_dirty(dn->dn_dbuf, tx);
|
|
|
|
|
|
|
|
|
|
dsl_dataset_dirty(os->os_dsl_dataset, tx);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
void
|
|
|
|
|
dnode_free(dnode_t *dn, dmu_tx_t *tx)
|
|
|
|
|
{
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
|
|
|
|
if (dn->dn_type == DMU_OT_NONE || dn->dn_free_txg) {
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
dn->dn_free_txg = tx->tx_txg;
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
|
2017-03-21 04:36:00 +03:00
|
|
|
dnode_setdirty(dn, tx);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Try to change the block size for the indicated dnode. This can only
|
|
|
|
|
* succeed if there are no blocks allocated or dirty beyond first block
|
|
|
|
|
*/
|
|
|
|
|
int
|
|
|
|
|
dnode_set_blksz(dnode_t *dn, uint64_t size, int ibs, dmu_tx_t *tx)
|
|
|
|
|
{
|
2015-04-03 06:14:28 +03:00
|
|
|
dmu_buf_impl_t *db;
|
2008-12-03 23:09:06 +03:00
|
|
|
int err;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2014-11-03 23:15:08 +03:00
|
|
|
ASSERT3U(size, <=, spa_maxblocksize(dmu_objset_spa(dn->dn_objset)));
|
2008-11-20 23:01:55 +03:00
|
|
|
if (size == 0)
|
|
|
|
|
size = SPA_MINBLOCKSIZE;
|
|
|
|
|
else
|
|
|
|
|
size = P2ROUNDUP(size, SPA_MINBLOCKSIZE);
|
|
|
|
|
|
|
|
|
|
if (ibs == dn->dn_indblkshift)
|
|
|
|
|
ibs = 0;
|
|
|
|
|
|
2023-07-15 02:16:40 +03:00
|
|
|
if (size == dn->dn_datablksz && ibs == 0)
|
2008-11-20 23:01:55 +03:00
|
|
|
return (0);
|
|
|
|
|
|
|
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_WRITER);
|
|
|
|
|
|
|
|
|
|
/* Check for any allocated blocks beyond the first */
|
Illumos #4101, #4102, #4103, #4105, #4106
4101 metaslab_debug should allow for fine-grained control
4102 space_maps should store more information about themselves
4103 space map object blocksize should be increased
4105 removing a mirrored log device results in a leaked object
4106 asynchronously load metaslab
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Sebastien Roy <seb@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Prior to this patch, space_maps were preferred solely based on the
amount of free space left in each. Unfortunately, this heuristic didn't
contain any information about the make-up of that free space, which
meant we could keep preferring and loading a highly fragmented space map
that wouldn't actually have enough contiguous space to satisfy the
allocation; then unloading that space_map and repeating the process.
This change modifies the space_map's to store additional information
about the contiguous space in the space_map, so that we can use this
information to make a better decision about which space_map to load.
This requires reallocating all space_map objects to increase their
bonus buffer size sizes enough to fit the new metadata.
The above feature can be enabled via a new feature flag introduced by
this change: com.delphix:spacemap_histogram
In addition to the above, this patch allows the space_map block size to
be increase. Currently the block size is set to be 4K in size, which has
certain implications including the following:
* 4K sector devices will not see any compression benefit
* large space_maps require more metadata on-disk
* large space_maps require more time to load (typically random reads)
Now the space_map block size can adjust as needed up to the maximum size
set via the space_map_max_blksz variable.
A bug was fixed which resulted in potentially leaking an object when
removing a mirrored log device. The previous logic for vdev_remove() did
not deal with removing top-level vdevs that are interior vdevs (i.e.
mirror) correctly. The problem would occur when removing a mirrored log
device, and result in the DTL space map object being leaked; because
top-level vdevs don't have DTL space map objects associated with them.
References:
https://www.illumos.org/issues/4101
https://www.illumos.org/issues/4102
https://www.illumos.org/issues/4103
https://www.illumos.org/issues/4105
https://www.illumos.org/issues/4106
https://github.com/illumos/illumos-gate/commit/0713e23
Porting notes:
A handful of kmem_alloc() calls were converted to kmem_zalloc(). Also,
the KM_PUSHPAGE and TQ_PUSHPAGE flags were used as necessary.
Ported-by: Tim Chase <tim@chase2k.com>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #2488
2013-10-02 01:25:53 +04:00
|
|
|
if (dn->dn_maxblkid != 0)
|
2008-11-20 23:01:55 +03:00
|
|
|
goto fail;
|
|
|
|
|
|
|
|
|
|
mutex_enter(&dn->dn_dbufs_mtx);
|
2015-04-03 06:14:28 +03:00
|
|
|
for (db = avl_first(&dn->dn_dbufs); db != NULL;
|
|
|
|
|
db = AVL_NEXT(&dn->dn_dbufs, db)) {
|
2010-05-29 00:45:14 +04:00
|
|
|
if (db->db_blkid != 0 && db->db_blkid != DMU_BONUS_BLKID &&
|
|
|
|
|
db->db_blkid != DMU_SPILL_BLKID) {
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_exit(&dn->dn_dbufs_mtx);
|
|
|
|
|
goto fail;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
mutex_exit(&dn->dn_dbufs_mtx);
|
|
|
|
|
|
|
|
|
|
if (ibs && dn->dn_nlevels != 1)
|
|
|
|
|
goto fail;
|
|
|
|
|
|
|
|
|
|
dnode_setdirty(dn, tx);
|
2023-07-15 02:16:40 +03:00
|
|
|
if (size != dn->dn_datablksz) {
|
|
|
|
|
/* resize the old block */
|
|
|
|
|
err = dbuf_hold_impl(dn, 0, 0, TRUE, FALSE, FTAG, &db);
|
|
|
|
|
if (err == 0) {
|
|
|
|
|
dbuf_new_size(db, size, tx);
|
|
|
|
|
} else if (err != ENOENT) {
|
|
|
|
|
goto fail;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
dnode_setdblksz(dn, size);
|
|
|
|
|
dn->dn_next_blksz[tx->tx_txg & TXG_MASK] = size;
|
|
|
|
|
if (db)
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
}
|
2008-11-20 23:01:55 +03:00
|
|
|
if (ibs) {
|
|
|
|
|
dn->dn_indblkshift = ibs;
|
2023-07-15 02:16:40 +03:00
|
|
|
dn->dn_next_indblkshift[tx->tx_txg & TXG_MASK] = ibs;
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
|
|
|
|
return (0);
|
|
|
|
|
|
|
|
|
|
fail:
|
|
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
2013-03-08 22:41:28 +04:00
|
|
|
return (SET_ERROR(ENOTSUP));
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
Native Encryption for ZFS on Linux
This change incorporates three major pieces:
The first change is a keystore that manages wrapping
and encryption keys for encrypted datasets. These
commands mostly involve manipulating the new
DSL Crypto Key ZAP Objects that live in the MOS. Each
encrypted dataset has its own DSL Crypto Key that is
protected with a user's key. This level of indirection
allows users to change their keys without re-encrypting
their entire datasets. The change implements the new
subcommands "zfs load-key", "zfs unload-key" and
"zfs change-key" which allow the user to manage their
encryption keys and settings. In addition, several new
flags and properties have been added to allow dataset
creation and to make mounting and unmounting more
convenient.
The second piece of this patch provides the ability to
encrypt, decyrpt, and authenticate protected datasets.
Each object set maintains a Merkel tree of Message
Authentication Codes that protect the lower layers,
similarly to how checksums are maintained. This part
impacts the zio layer, which handles the actual
encryption and generation of MACs, as well as the ARC
and DMU, which need to be able to handle encrypted
buffers and protected data.
The last addition is the ability to do raw, encrypted
sends and receives. The idea here is to send raw
encrypted and compressed data and receive it exactly
as is on a backup system. This means that the dataset
on the receiving system is protected using the same
user key that is in use on the sending side. By doing
so, datasets can be efficiently backed up to an
untrusted system without fear of data being
compromised.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #494
Closes #5769
2017-08-14 20:36:48 +03:00
|
|
|
static void
|
|
|
|
|
dnode_set_nlevels_impl(dnode_t *dn, int new_nlevels, dmu_tx_t *tx)
|
|
|
|
|
{
|
|
|
|
|
uint64_t txgoff = tx->tx_txg & TXG_MASK;
|
|
|
|
|
int old_nlevels = dn->dn_nlevels;
|
|
|
|
|
dmu_buf_impl_t *db;
|
|
|
|
|
list_t *list;
|
|
|
|
|
dbuf_dirty_record_t *new, *dr, *dr_next;
|
|
|
|
|
|
|
|
|
|
ASSERT(RW_WRITE_HELD(&dn->dn_struct_rwlock));
|
|
|
|
|
|
Improve zfs receive performance with lightweight write
The performance of `zfs receive` can be bottlenecked on the CPU consumed
by the `receive_writer` thread, especially when receiving streams with
small compressed block sizes. Much of the CPU is spent creating and
destroying dbuf's and arc buf's, one for each `WRITE` record in the send
stream.
This commit introduces the concept of "lightweight writes", which allows
`zfs receive` to write to the DMU by providing an ABD, and instantiating
only a new type of `dbuf_dirty_record_t`. The dbuf and arc buf for this
"dirty leaf block" are not instantiated.
Because there is no dbuf with the dirty data, this mechanism doesn't
support reading from "lightweight-dirty" blocks (they would see the
on-disk state rather than the dirty data). Since the dedup-receive code
has been removed, `zfs receive` is write-only, so this works fine.
Because there are no arc bufs for the received data, the received data
is no longer cached in the ARC.
Testing a receive of a stream with average compressed block size of 4KB,
this commit improves performance by 50%, while also reducing CPU usage
by 50% of a CPU. On a per-block basis, CPU consumed by receive_writer()
and dbuf_evict() is now 1/7th (14%) of what it was.
Baseline: 450MB/s, CPU in receive_writer() 40% + dbuf_evict() 35%
New: 670MB/s, CPU in receive_writer() 17% + dbuf_evict() 0%
The code is also restructured in a few ways:
Added a `dr_dnode` field to the dbuf_dirty_record_t. This simplifies
some existing code that no longer needs `DB_DNODE_ENTER()` and related
routines. The new field is needed by the lightweight-type dirty record.
To ensure that the `dr_dnode` field remains valid until the dirty record
is freed, we have to ensure that the `dnode_move()` doesn't relocate the
dnode_t. To do this we keep a hold on the dnode until it's zio's have
completed. This is already done by the user-accounting code
(`userquota_updates_task()`), this commit extends that so that it always
keeps the dnode hold until zio completion (see `dnode_rele_task()`).
`dn_dirty_txg` was previously zeroed when the dnode was synced. This
was not necessary, since its meaning can be "when was this dnode last
dirtied". This change simplifies the new `dnode_rele_task()` code.
Removed some dead code related to `DRR_WRITE_BYREF` (dedup receive).
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #11105
2020-12-11 21:26:02 +03:00
|
|
|
ASSERT3U(new_nlevels, >, dn->dn_nlevels);
|
Native Encryption for ZFS on Linux
This change incorporates three major pieces:
The first change is a keystore that manages wrapping
and encryption keys for encrypted datasets. These
commands mostly involve manipulating the new
DSL Crypto Key ZAP Objects that live in the MOS. Each
encrypted dataset has its own DSL Crypto Key that is
protected with a user's key. This level of indirection
allows users to change their keys without re-encrypting
their entire datasets. The change implements the new
subcommands "zfs load-key", "zfs unload-key" and
"zfs change-key" which allow the user to manage their
encryption keys and settings. In addition, several new
flags and properties have been added to allow dataset
creation and to make mounting and unmounting more
convenient.
The second piece of this patch provides the ability to
encrypt, decyrpt, and authenticate protected datasets.
Each object set maintains a Merkel tree of Message
Authentication Codes that protect the lower layers,
similarly to how checksums are maintained. This part
impacts the zio layer, which handles the actual
encryption and generation of MACs, as well as the ARC
and DMU, which need to be able to handle encrypted
buffers and protected data.
The last addition is the ability to do raw, encrypted
sends and receives. The idea here is to send raw
encrypted and compressed data and receive it exactly
as is on a backup system. This means that the dataset
on the receiving system is protected using the same
user key that is in use on the sending side. By doing
so, datasets can be efficiently backed up to an
untrusted system without fear of data being
compromised.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #494
Closes #5769
2017-08-14 20:36:48 +03:00
|
|
|
dn->dn_nlevels = new_nlevels;
|
|
|
|
|
|
|
|
|
|
ASSERT3U(new_nlevels, >, dn->dn_next_nlevels[txgoff]);
|
|
|
|
|
dn->dn_next_nlevels[txgoff] = new_nlevels;
|
|
|
|
|
|
|
|
|
|
/* dirty the left indirects */
|
|
|
|
|
db = dbuf_hold_level(dn, old_nlevels, 0, FTAG);
|
|
|
|
|
ASSERT(db != NULL);
|
|
|
|
|
new = dbuf_dirty(db, tx);
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
|
|
|
|
|
/* transfer the dirty records to the new indirect */
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
|
|
|
|
mutex_enter(&new->dt.di.dr_mtx);
|
|
|
|
|
list = &dn->dn_dirty_records[txgoff];
|
|
|
|
|
for (dr = list_head(list); dr; dr = dr_next) {
|
|
|
|
|
dr_next = list_next(&dn->dn_dirty_records[txgoff], dr);
|
Improve zfs receive performance with lightweight write
The performance of `zfs receive` can be bottlenecked on the CPU consumed
by the `receive_writer` thread, especially when receiving streams with
small compressed block sizes. Much of the CPU is spent creating and
destroying dbuf's and arc buf's, one for each `WRITE` record in the send
stream.
This commit introduces the concept of "lightweight writes", which allows
`zfs receive` to write to the DMU by providing an ABD, and instantiating
only a new type of `dbuf_dirty_record_t`. The dbuf and arc buf for this
"dirty leaf block" are not instantiated.
Because there is no dbuf with the dirty data, this mechanism doesn't
support reading from "lightweight-dirty" blocks (they would see the
on-disk state rather than the dirty data). Since the dedup-receive code
has been removed, `zfs receive` is write-only, so this works fine.
Because there are no arc bufs for the received data, the received data
is no longer cached in the ARC.
Testing a receive of a stream with average compressed block size of 4KB,
this commit improves performance by 50%, while also reducing CPU usage
by 50% of a CPU. On a per-block basis, CPU consumed by receive_writer()
and dbuf_evict() is now 1/7th (14%) of what it was.
Baseline: 450MB/s, CPU in receive_writer() 40% + dbuf_evict() 35%
New: 670MB/s, CPU in receive_writer() 17% + dbuf_evict() 0%
The code is also restructured in a few ways:
Added a `dr_dnode` field to the dbuf_dirty_record_t. This simplifies
some existing code that no longer needs `DB_DNODE_ENTER()` and related
routines. The new field is needed by the lightweight-type dirty record.
To ensure that the `dr_dnode` field remains valid until the dirty record
is freed, we have to ensure that the `dnode_move()` doesn't relocate the
dnode_t. To do this we keep a hold on the dnode until it's zio's have
completed. This is already done by the user-accounting code
(`userquota_updates_task()`), this commit extends that so that it always
keeps the dnode hold until zio completion (see `dnode_rele_task()`).
`dn_dirty_txg` was previously zeroed when the dnode was synced. This
was not necessary, since its meaning can be "when was this dnode last
dirtied". This change simplifies the new `dnode_rele_task()` code.
Removed some dead code related to `DRR_WRITE_BYREF` (dedup receive).
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #11105
2020-12-11 21:26:02 +03:00
|
|
|
|
|
|
|
|
IMPLY(dr->dr_dbuf == NULL, old_nlevels == 1);
|
|
|
|
|
if (dr->dr_dbuf == NULL ||
|
|
|
|
|
(dr->dr_dbuf->db_level == old_nlevels - 1 &&
|
Native Encryption for ZFS on Linux
This change incorporates three major pieces:
The first change is a keystore that manages wrapping
and encryption keys for encrypted datasets. These
commands mostly involve manipulating the new
DSL Crypto Key ZAP Objects that live in the MOS. Each
encrypted dataset has its own DSL Crypto Key that is
protected with a user's key. This level of indirection
allows users to change their keys without re-encrypting
their entire datasets. The change implements the new
subcommands "zfs load-key", "zfs unload-key" and
"zfs change-key" which allow the user to manage their
encryption keys and settings. In addition, several new
flags and properties have been added to allow dataset
creation and to make mounting and unmounting more
convenient.
The second piece of this patch provides the ability to
encrypt, decyrpt, and authenticate protected datasets.
Each object set maintains a Merkel tree of Message
Authentication Codes that protect the lower layers,
similarly to how checksums are maintained. This part
impacts the zio layer, which handles the actual
encryption and generation of MACs, as well as the ARC
and DMU, which need to be able to handle encrypted
buffers and protected data.
The last addition is the ability to do raw, encrypted
sends and receives. The idea here is to send raw
encrypted and compressed data and receive it exactly
as is on a backup system. This means that the dataset
on the receiving system is protected using the same
user key that is in use on the sending side. By doing
so, datasets can be efficiently backed up to an
untrusted system without fear of data being
compromised.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #494
Closes #5769
2017-08-14 20:36:48 +03:00
|
|
|
dr->dr_dbuf->db_blkid != DMU_BONUS_BLKID &&
|
Improve zfs receive performance with lightweight write
The performance of `zfs receive` can be bottlenecked on the CPU consumed
by the `receive_writer` thread, especially when receiving streams with
small compressed block sizes. Much of the CPU is spent creating and
destroying dbuf's and arc buf's, one for each `WRITE` record in the send
stream.
This commit introduces the concept of "lightweight writes", which allows
`zfs receive` to write to the DMU by providing an ABD, and instantiating
only a new type of `dbuf_dirty_record_t`. The dbuf and arc buf for this
"dirty leaf block" are not instantiated.
Because there is no dbuf with the dirty data, this mechanism doesn't
support reading from "lightweight-dirty" blocks (they would see the
on-disk state rather than the dirty data). Since the dedup-receive code
has been removed, `zfs receive` is write-only, so this works fine.
Because there are no arc bufs for the received data, the received data
is no longer cached in the ARC.
Testing a receive of a stream with average compressed block size of 4KB,
this commit improves performance by 50%, while also reducing CPU usage
by 50% of a CPU. On a per-block basis, CPU consumed by receive_writer()
and dbuf_evict() is now 1/7th (14%) of what it was.
Baseline: 450MB/s, CPU in receive_writer() 40% + dbuf_evict() 35%
New: 670MB/s, CPU in receive_writer() 17% + dbuf_evict() 0%
The code is also restructured in a few ways:
Added a `dr_dnode` field to the dbuf_dirty_record_t. This simplifies
some existing code that no longer needs `DB_DNODE_ENTER()` and related
routines. The new field is needed by the lightweight-type dirty record.
To ensure that the `dr_dnode` field remains valid until the dirty record
is freed, we have to ensure that the `dnode_move()` doesn't relocate the
dnode_t. To do this we keep a hold on the dnode until it's zio's have
completed. This is already done by the user-accounting code
(`userquota_updates_task()`), this commit extends that so that it always
keeps the dnode hold until zio completion (see `dnode_rele_task()`).
`dn_dirty_txg` was previously zeroed when the dnode was synced. This
was not necessary, since its meaning can be "when was this dnode last
dirtied". This change simplifies the new `dnode_rele_task()` code.
Removed some dead code related to `DRR_WRITE_BYREF` (dedup receive).
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Paul Dagnelie <pcd@delphix.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #11105
2020-12-11 21:26:02 +03:00
|
|
|
dr->dr_dbuf->db_blkid != DMU_SPILL_BLKID)) {
|
Native Encryption for ZFS on Linux
This change incorporates three major pieces:
The first change is a keystore that manages wrapping
and encryption keys for encrypted datasets. These
commands mostly involve manipulating the new
DSL Crypto Key ZAP Objects that live in the MOS. Each
encrypted dataset has its own DSL Crypto Key that is
protected with a user's key. This level of indirection
allows users to change their keys without re-encrypting
their entire datasets. The change implements the new
subcommands "zfs load-key", "zfs unload-key" and
"zfs change-key" which allow the user to manage their
encryption keys and settings. In addition, several new
flags and properties have been added to allow dataset
creation and to make mounting and unmounting more
convenient.
The second piece of this patch provides the ability to
encrypt, decyrpt, and authenticate protected datasets.
Each object set maintains a Merkel tree of Message
Authentication Codes that protect the lower layers,
similarly to how checksums are maintained. This part
impacts the zio layer, which handles the actual
encryption and generation of MACs, as well as the ARC
and DMU, which need to be able to handle encrypted
buffers and protected data.
The last addition is the ability to do raw, encrypted
sends and receives. The idea here is to send raw
encrypted and compressed data and receive it exactly
as is on a backup system. This means that the dataset
on the receiving system is protected using the same
user key that is in use on the sending side. By doing
so, datasets can be efficiently backed up to an
untrusted system without fear of data being
compromised.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #494
Closes #5769
2017-08-14 20:36:48 +03:00
|
|
|
list_remove(&dn->dn_dirty_records[txgoff], dr);
|
|
|
|
|
list_insert_tail(&new->dt.di.dr_children, dr);
|
|
|
|
|
dr->dr_parent = new;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
mutex_exit(&new->dt.di.dr_mtx);
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
int
|
|
|
|
|
dnode_set_nlevels(dnode_t *dn, int nlevels, dmu_tx_t *tx)
|
|
|
|
|
{
|
|
|
|
|
int ret = 0;
|
|
|
|
|
|
|
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_WRITER);
|
|
|
|
|
|
|
|
|
|
if (dn->dn_nlevels == nlevels) {
|
|
|
|
|
ret = 0;
|
|
|
|
|
goto out;
|
|
|
|
|
} else if (nlevels < dn->dn_nlevels) {
|
|
|
|
|
ret = SET_ERROR(EINVAL);
|
|
|
|
|
goto out;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
dnode_set_nlevels_impl(dn, nlevels, tx);
|
|
|
|
|
|
|
|
|
|
out:
|
|
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
|
|
|
|
return (ret);
|
|
|
|
|
}
|
|
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
/* read-holding callers must not rely on the lock being continuously held */
|
2008-11-20 23:01:55 +03:00
|
|
|
void
|
2019-03-13 20:52:01 +03:00
|
|
|
dnode_new_blkid(dnode_t *dn, uint64_t blkid, dmu_tx_t *tx, boolean_t have_read,
|
|
|
|
|
boolean_t force)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
|
|
|
|
int epbs, new_nlevels;
|
|
|
|
|
uint64_t sz;
|
|
|
|
|
|
2010-05-29 00:45:14 +04:00
|
|
|
ASSERT(blkid != DMU_BONUS_BLKID);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
ASSERT(have_read ?
|
|
|
|
|
RW_READ_HELD(&dn->dn_struct_rwlock) :
|
|
|
|
|
RW_WRITE_HELD(&dn->dn_struct_rwlock));
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* if we have a read-lock, check to see if we need to do any work
|
|
|
|
|
* before upgrading to a write-lock.
|
|
|
|
|
*/
|
|
|
|
|
if (have_read) {
|
|
|
|
|
if (blkid <= dn->dn_maxblkid)
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
if (!rw_tryupgrade(&dn->dn_struct_rwlock)) {
|
|
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
|
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_WRITER);
|
|
|
|
|
}
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
2019-03-13 20:52:01 +03:00
|
|
|
/*
|
|
|
|
|
* Raw sends (indicated by the force flag) require that we take the
|
|
|
|
|
* given blkid even if the value is lower than the current value.
|
|
|
|
|
*/
|
|
|
|
|
if (!force && blkid <= dn->dn_maxblkid)
|
2008-11-20 23:01:55 +03:00
|
|
|
goto out;
|
|
|
|
|
|
2019-03-13 20:52:01 +03:00
|
|
|
/*
|
|
|
|
|
* We use the (otherwise unused) top bit of dn_next_maxblkid[txgoff]
|
|
|
|
|
* to indicate that this field is set. This allows us to set the
|
|
|
|
|
* maxblkid to 0 on an existing object in dnode_sync().
|
|
|
|
|
*/
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_maxblkid = blkid;
|
2019-03-13 20:52:01 +03:00
|
|
|
dn->dn_next_maxblkid[tx->tx_txg & TXG_MASK] =
|
|
|
|
|
blkid | DMU_NEXT_MAXBLKID_SET;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Compute the number of levels necessary to support the new maxblkid.
|
2019-03-13 20:52:01 +03:00
|
|
|
* Raw sends will ensure nlevels is set correctly for us.
|
2008-11-20 23:01:55 +03:00
|
|
|
*/
|
|
|
|
|
new_nlevels = 1;
|
|
|
|
|
epbs = dn->dn_indblkshift - SPA_BLKPTRSHIFT;
|
|
|
|
|
for (sz = dn->dn_nblkptr;
|
|
|
|
|
sz <= blkid && sz >= dn->dn_nblkptr; sz <<= epbs)
|
|
|
|
|
new_nlevels++;
|
|
|
|
|
|
2016-08-31 11:12:08 +03:00
|
|
|
ASSERT3U(new_nlevels, <=, DN_MAX_LEVELS);
|
|
|
|
|
|
2019-03-13 20:52:01 +03:00
|
|
|
if (!force) {
|
|
|
|
|
if (new_nlevels > dn->dn_nlevels)
|
|
|
|
|
dnode_set_nlevels_impl(dn, new_nlevels, tx);
|
|
|
|
|
} else {
|
|
|
|
|
ASSERT3U(dn->dn_nlevels, >=, new_nlevels);
|
|
|
|
|
}
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
out:
|
2008-12-03 23:09:06 +03:00
|
|
|
if (have_read)
|
|
|
|
|
rw_downgrade(&dn->dn_struct_rwlock);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
2015-07-02 19:23:20 +03:00
|
|
|
static void
|
|
|
|
|
dnode_dirty_l1(dnode_t *dn, uint64_t l1blkid, dmu_tx_t *tx)
|
|
|
|
|
{
|
|
|
|
|
dmu_buf_impl_t *db = dbuf_hold_level(dn, 1, l1blkid, FTAG);
|
|
|
|
|
if (db != NULL) {
|
|
|
|
|
dmu_buf_will_dirty(&db->db, tx);
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
OpenZFS 9438 - Holes can lose birth time info if a block has a mix of birth times
As reported by https://github.com/zfsonlinux/zfs/issues/4996, there is
yet another hole birth issue. In this one, if a block is entirely holes,
but the birth times are not all the same, we lose that information by
creating one hole with the current txg as its birth time.
The ZoL PR's fix approach is incorrect. Ultimately, the problem here is
that when you truncate and write a file in the same transaction group,
the dbuf for the indirect block will be zeroed out to deal with the
truncation, and then written for the write. During this process, we will
lose hole birth time information for any holes in the range. In the case
where a dnode is being freed, we need to determine whether the block
should be converted to a higher-level hole in the zio pipeline, and if
so do it when the dnode is being synced out.
Porting Notes:
* The DMU_OBJECT_END change in zfs_znode.c was already applied.
* Added test cases from #5675 provided by @rincebrain for hole_birth
issues. These test cases should be pushed upstream to OpenZFS.
* Updated mk_files which is used by several rsend tests so the
files created are a little more interesting and may contain holes.
Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/9438
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/738e2a3c
External-issue: DLPX-46861
Closes #7746
2016-09-20 20:02:29 +03:00
|
|
|
/*
|
|
|
|
|
* Dirty all the in-core level-1 dbufs in the range specified by start_blkid
|
|
|
|
|
* and end_blkid.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
dnode_dirty_l1range(dnode_t *dn, uint64_t start_blkid, uint64_t end_blkid,
|
|
|
|
|
dmu_tx_t *tx)
|
|
|
|
|
{
|
2020-09-07 18:33:34 +03:00
|
|
|
dmu_buf_impl_t *db_search;
|
OpenZFS 9438 - Holes can lose birth time info if a block has a mix of birth times
As reported by https://github.com/zfsonlinux/zfs/issues/4996, there is
yet another hole birth issue. In this one, if a block is entirely holes,
but the birth times are not all the same, we lose that information by
creating one hole with the current txg as its birth time.
The ZoL PR's fix approach is incorrect. Ultimately, the problem here is
that when you truncate and write a file in the same transaction group,
the dbuf for the indirect block will be zeroed out to deal with the
truncation, and then written for the write. During this process, we will
lose hole birth time information for any holes in the range. In the case
where a dnode is being freed, we need to determine whether the block
should be converted to a higher-level hole in the zio pipeline, and if
so do it when the dnode is being synced out.
Porting Notes:
* The DMU_OBJECT_END change in zfs_znode.c was already applied.
* Added test cases from #5675 provided by @rincebrain for hole_birth
issues. These test cases should be pushed upstream to OpenZFS.
* Updated mk_files which is used by several rsend tests so the
files created are a little more interesting and may contain holes.
Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/9438
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/738e2a3c
External-issue: DLPX-46861
Closes #7746
2016-09-20 20:02:29 +03:00
|
|
|
dmu_buf_impl_t *db;
|
|
|
|
|
avl_index_t where;
|
|
|
|
|
|
2020-09-07 18:33:34 +03:00
|
|
|
db_search = kmem_zalloc(sizeof (dmu_buf_impl_t), KM_SLEEP);
|
|
|
|
|
|
OpenZFS 9438 - Holes can lose birth time info if a block has a mix of birth times
As reported by https://github.com/zfsonlinux/zfs/issues/4996, there is
yet another hole birth issue. In this one, if a block is entirely holes,
but the birth times are not all the same, we lose that information by
creating one hole with the current txg as its birth time.
The ZoL PR's fix approach is incorrect. Ultimately, the problem here is
that when you truncate and write a file in the same transaction group,
the dbuf for the indirect block will be zeroed out to deal with the
truncation, and then written for the write. During this process, we will
lose hole birth time information for any holes in the range. In the case
where a dnode is being freed, we need to determine whether the block
should be converted to a higher-level hole in the zio pipeline, and if
so do it when the dnode is being synced out.
Porting Notes:
* The DMU_OBJECT_END change in zfs_znode.c was already applied.
* Added test cases from #5675 provided by @rincebrain for hole_birth
issues. These test cases should be pushed upstream to OpenZFS.
* Updated mk_files which is used by several rsend tests so the
files created are a little more interesting and may contain holes.
Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/9438
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/738e2a3c
External-issue: DLPX-46861
Closes #7746
2016-09-20 20:02:29 +03:00
|
|
|
mutex_enter(&dn->dn_dbufs_mtx);
|
|
|
|
|
|
2020-09-07 18:33:34 +03:00
|
|
|
db_search->db_level = 1;
|
|
|
|
|
db_search->db_blkid = start_blkid + 1;
|
|
|
|
|
db_search->db_state = DB_SEARCH;
|
OpenZFS 9438 - Holes can lose birth time info if a block has a mix of birth times
As reported by https://github.com/zfsonlinux/zfs/issues/4996, there is
yet another hole birth issue. In this one, if a block is entirely holes,
but the birth times are not all the same, we lose that information by
creating one hole with the current txg as its birth time.
The ZoL PR's fix approach is incorrect. Ultimately, the problem here is
that when you truncate and write a file in the same transaction group,
the dbuf for the indirect block will be zeroed out to deal with the
truncation, and then written for the write. During this process, we will
lose hole birth time information for any holes in the range. In the case
where a dnode is being freed, we need to determine whether the block
should be converted to a higher-level hole in the zio pipeline, and if
so do it when the dnode is being synced out.
Porting Notes:
* The DMU_OBJECT_END change in zfs_znode.c was already applied.
* Added test cases from #5675 provided by @rincebrain for hole_birth
issues. These test cases should be pushed upstream to OpenZFS.
* Updated mk_files which is used by several rsend tests so the
files created are a little more interesting and may contain holes.
Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/9438
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/738e2a3c
External-issue: DLPX-46861
Closes #7746
2016-09-20 20:02:29 +03:00
|
|
|
for (;;) {
|
|
|
|
|
|
2020-09-07 18:33:34 +03:00
|
|
|
db = avl_find(&dn->dn_dbufs, db_search, &where);
|
OpenZFS 9438 - Holes can lose birth time info if a block has a mix of birth times
As reported by https://github.com/zfsonlinux/zfs/issues/4996, there is
yet another hole birth issue. In this one, if a block is entirely holes,
but the birth times are not all the same, we lose that information by
creating one hole with the current txg as its birth time.
The ZoL PR's fix approach is incorrect. Ultimately, the problem here is
that when you truncate and write a file in the same transaction group,
the dbuf for the indirect block will be zeroed out to deal with the
truncation, and then written for the write. During this process, we will
lose hole birth time information for any holes in the range. In the case
where a dnode is being freed, we need to determine whether the block
should be converted to a higher-level hole in the zio pipeline, and if
so do it when the dnode is being synced out.
Porting Notes:
* The DMU_OBJECT_END change in zfs_znode.c was already applied.
* Added test cases from #5675 provided by @rincebrain for hole_birth
issues. These test cases should be pushed upstream to OpenZFS.
* Updated mk_files which is used by several rsend tests so the
files created are a little more interesting and may contain holes.
Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/9438
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/738e2a3c
External-issue: DLPX-46861
Closes #7746
2016-09-20 20:02:29 +03:00
|
|
|
if (db == NULL)
|
|
|
|
|
db = avl_nearest(&dn->dn_dbufs, where, AVL_AFTER);
|
|
|
|
|
|
|
|
|
|
if (db == NULL || db->db_level != 1 ||
|
|
|
|
|
db->db_blkid >= end_blkid) {
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Setup the next blkid we want to search for.
|
|
|
|
|
*/
|
2020-09-07 18:33:34 +03:00
|
|
|
db_search->db_blkid = db->db_blkid + 1;
|
OpenZFS 9438 - Holes can lose birth time info if a block has a mix of birth times
As reported by https://github.com/zfsonlinux/zfs/issues/4996, there is
yet another hole birth issue. In this one, if a block is entirely holes,
but the birth times are not all the same, we lose that information by
creating one hole with the current txg as its birth time.
The ZoL PR's fix approach is incorrect. Ultimately, the problem here is
that when you truncate and write a file in the same transaction group,
the dbuf for the indirect block will be zeroed out to deal with the
truncation, and then written for the write. During this process, we will
lose hole birth time information for any holes in the range. In the case
where a dnode is being freed, we need to determine whether the block
should be converted to a higher-level hole in the zio pipeline, and if
so do it when the dnode is being synced out.
Porting Notes:
* The DMU_OBJECT_END change in zfs_znode.c was already applied.
* Added test cases from #5675 provided by @rincebrain for hole_birth
issues. These test cases should be pushed upstream to OpenZFS.
* Updated mk_files which is used by several rsend tests so the
files created are a little more interesting and may contain holes.
Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/9438
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/738e2a3c
External-issue: DLPX-46861
Closes #7746
2016-09-20 20:02:29 +03:00
|
|
|
ASSERT3U(db->db_blkid, >=, start_blkid);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If the dbuf transitions to DB_EVICTING while we're trying
|
|
|
|
|
* to dirty it, then we will be unable to discover it in
|
|
|
|
|
* the dbuf hash table. This will result in a call to
|
|
|
|
|
* dbuf_create() which needs to acquire the dn_dbufs_mtx
|
|
|
|
|
* lock. To avoid a deadlock, we drop the lock before
|
|
|
|
|
* dirtying the level-1 dbuf.
|
|
|
|
|
*/
|
|
|
|
|
mutex_exit(&dn->dn_dbufs_mtx);
|
|
|
|
|
dnode_dirty_l1(dn, db->db_blkid, tx);
|
|
|
|
|
mutex_enter(&dn->dn_dbufs_mtx);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#ifdef ZFS_DEBUG
|
|
|
|
|
/*
|
|
|
|
|
* Walk all the in-core level-1 dbufs and verify they have been dirtied.
|
|
|
|
|
*/
|
2020-09-07 18:33:34 +03:00
|
|
|
db_search->db_level = 1;
|
|
|
|
|
db_search->db_blkid = start_blkid + 1;
|
|
|
|
|
db_search->db_state = DB_SEARCH;
|
|
|
|
|
db = avl_find(&dn->dn_dbufs, db_search, &where);
|
OpenZFS 9438 - Holes can lose birth time info if a block has a mix of birth times
As reported by https://github.com/zfsonlinux/zfs/issues/4996, there is
yet another hole birth issue. In this one, if a block is entirely holes,
but the birth times are not all the same, we lose that information by
creating one hole with the current txg as its birth time.
The ZoL PR's fix approach is incorrect. Ultimately, the problem here is
that when you truncate and write a file in the same transaction group,
the dbuf for the indirect block will be zeroed out to deal with the
truncation, and then written for the write. During this process, we will
lose hole birth time information for any holes in the range. In the case
where a dnode is being freed, we need to determine whether the block
should be converted to a higher-level hole in the zio pipeline, and if
so do it when the dnode is being synced out.
Porting Notes:
* The DMU_OBJECT_END change in zfs_znode.c was already applied.
* Added test cases from #5675 provided by @rincebrain for hole_birth
issues. These test cases should be pushed upstream to OpenZFS.
* Updated mk_files which is used by several rsend tests so the
files created are a little more interesting and may contain holes.
Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/9438
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/738e2a3c
External-issue: DLPX-46861
Closes #7746
2016-09-20 20:02:29 +03:00
|
|
|
if (db == NULL)
|
|
|
|
|
db = avl_nearest(&dn->dn_dbufs, where, AVL_AFTER);
|
|
|
|
|
for (; db != NULL; db = AVL_NEXT(&dn->dn_dbufs, db)) {
|
|
|
|
|
if (db->db_level != 1 || db->db_blkid >= end_blkid)
|
|
|
|
|
break;
|
2019-05-20 03:30:33 +03:00
|
|
|
if (db->db_state != DB_EVICTING)
|
|
|
|
|
ASSERT(db->db_dirtycnt > 0);
|
OpenZFS 9438 - Holes can lose birth time info if a block has a mix of birth times
As reported by https://github.com/zfsonlinux/zfs/issues/4996, there is
yet another hole birth issue. In this one, if a block is entirely holes,
but the birth times are not all the same, we lose that information by
creating one hole with the current txg as its birth time.
The ZoL PR's fix approach is incorrect. Ultimately, the problem here is
that when you truncate and write a file in the same transaction group,
the dbuf for the indirect block will be zeroed out to deal with the
truncation, and then written for the write. During this process, we will
lose hole birth time information for any holes in the range. In the case
where a dnode is being freed, we need to determine whether the block
should be converted to a higher-level hole in the zio pipeline, and if
so do it when the dnode is being synced out.
Porting Notes:
* The DMU_OBJECT_END change in zfs_znode.c was already applied.
* Added test cases from #5675 provided by @rincebrain for hole_birth
issues. These test cases should be pushed upstream to OpenZFS.
* Updated mk_files which is used by several rsend tests so the
files created are a little more interesting and may contain holes.
Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/9438
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/738e2a3c
External-issue: DLPX-46861
Closes #7746
2016-09-20 20:02:29 +03:00
|
|
|
}
|
|
|
|
|
#endif
|
2020-09-07 18:33:34 +03:00
|
|
|
kmem_free(db_search, sizeof (dmu_buf_impl_t));
|
OpenZFS 9438 - Holes can lose birth time info if a block has a mix of birth times
As reported by https://github.com/zfsonlinux/zfs/issues/4996, there is
yet another hole birth issue. In this one, if a block is entirely holes,
but the birth times are not all the same, we lose that information by
creating one hole with the current txg as its birth time.
The ZoL PR's fix approach is incorrect. Ultimately, the problem here is
that when you truncate and write a file in the same transaction group,
the dbuf for the indirect block will be zeroed out to deal with the
truncation, and then written for the write. During this process, we will
lose hole birth time information for any holes in the range. In the case
where a dnode is being freed, we need to determine whether the block
should be converted to a higher-level hole in the zio pipeline, and if
so do it when the dnode is being synced out.
Porting Notes:
* The DMU_OBJECT_END change in zfs_znode.c was already applied.
* Added test cases from #5675 provided by @rincebrain for hole_birth
issues. These test cases should be pushed upstream to OpenZFS.
* Updated mk_files which is used by several rsend tests so the
files created are a little more interesting and may contain holes.
Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/9438
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/738e2a3c
External-issue: DLPX-46861
Closes #7746
2016-09-20 20:02:29 +03:00
|
|
|
mutex_exit(&dn->dn_dbufs_mtx);
|
|
|
|
|
}
|
|
|
|
|
|
2020-02-27 03:09:17 +03:00
|
|
|
void
|
2022-04-19 21:38:30 +03:00
|
|
|
dnode_set_dirtyctx(dnode_t *dn, dmu_tx_t *tx, const void *tag)
|
2020-02-27 03:09:17 +03:00
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* Don't set dirtyctx to SYNC if we're just modifying this as we
|
|
|
|
|
* initialize the objset.
|
|
|
|
|
*/
|
|
|
|
|
if (dn->dn_dirtyctx == DN_UNDIRTIED) {
|
|
|
|
|
dsl_dataset_t *ds = dn->dn_objset->os_dsl_dataset;
|
|
|
|
|
|
|
|
|
|
if (ds != NULL) {
|
|
|
|
|
rrw_enter(&ds->ds_bp_rwlock, RW_READER, tag);
|
|
|
|
|
}
|
|
|
|
|
if (!BP_IS_HOLE(dn->dn_objset->os_rootbp)) {
|
|
|
|
|
if (dmu_tx_is_syncing(tx))
|
|
|
|
|
dn->dn_dirtyctx = DN_DIRTY_SYNC;
|
|
|
|
|
else
|
|
|
|
|
dn->dn_dirtyctx = DN_DIRTY_OPEN;
|
|
|
|
|
dn->dn_dirtyctx_firstset = tag;
|
|
|
|
|
}
|
|
|
|
|
if (ds != NULL) {
|
|
|
|
|
rrw_exit(&ds->ds_bp_rwlock, tag);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2021-10-19 02:50:33 +03:00
|
|
|
static void
|
|
|
|
|
dnode_partial_zero(dnode_t *dn, uint64_t off, uint64_t blkoff, uint64_t len,
|
|
|
|
|
dmu_tx_t *tx)
|
|
|
|
|
{
|
|
|
|
|
dmu_buf_impl_t *db;
|
|
|
|
|
int res;
|
|
|
|
|
|
|
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_READER);
|
|
|
|
|
res = dbuf_hold_impl(dn, 0, dbuf_whichblock(dn, 0, off), TRUE, FALSE,
|
|
|
|
|
FTAG, &db);
|
|
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
|
|
|
|
if (res == 0) {
|
|
|
|
|
db_lock_type_t dblt;
|
|
|
|
|
boolean_t dirty;
|
|
|
|
|
|
|
|
|
|
dblt = dmu_buf_lock_parent(db, RW_READER, FTAG);
|
|
|
|
|
/* don't dirty if not on disk and not dirty */
|
|
|
|
|
dirty = !list_is_empty(&db->db_dirty_records) ||
|
|
|
|
|
(db->db_blkptr && !BP_IS_HOLE(db->db_blkptr));
|
|
|
|
|
dmu_buf_unlock_parent(db, dblt, FTAG);
|
|
|
|
|
if (dirty) {
|
|
|
|
|
caddr_t data;
|
|
|
|
|
|
|
|
|
|
dmu_buf_will_dirty(&db->db, tx);
|
|
|
|
|
data = db->db.db_data;
|
2022-02-25 16:26:54 +03:00
|
|
|
memset(data + blkoff, 0, len);
|
2021-10-19 02:50:33 +03:00
|
|
|
}
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
void
|
|
|
|
|
dnode_free_range(dnode_t *dn, uint64_t off, uint64_t len, dmu_tx_t *tx)
|
|
|
|
|
{
|
|
|
|
|
uint64_t blkoff, blkid, nblks;
|
2008-12-03 23:09:06 +03:00
|
|
|
int blksz, blkshift, head, tail;
|
2008-11-20 23:01:55 +03:00
|
|
|
int trunc = FALSE;
|
2008-12-03 23:09:06 +03:00
|
|
|
int epbs;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
blksz = dn->dn_datablksz;
|
2008-12-03 23:09:06 +03:00
|
|
|
blkshift = dn->dn_datablkshift;
|
|
|
|
|
epbs = dn->dn_indblkshift - SPA_BLKPTRSHIFT;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2013-08-21 08:11:52 +04:00
|
|
|
if (len == DMU_OBJECT_END) {
|
2008-11-20 23:01:55 +03:00
|
|
|
len = UINT64_MAX - off;
|
|
|
|
|
trunc = TRUE;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* First, block align the region to free:
|
|
|
|
|
*/
|
|
|
|
|
if (ISP2(blksz)) {
|
|
|
|
|
head = P2NPHASE(off, blksz);
|
|
|
|
|
blkoff = P2PHASE(off, blksz);
|
2008-12-03 23:09:06 +03:00
|
|
|
if ((off >> blkshift) > dn->dn_maxblkid)
|
2019-07-08 23:18:50 +03:00
|
|
|
return;
|
2008-11-20 23:01:55 +03:00
|
|
|
} else {
|
|
|
|
|
ASSERT(dn->dn_maxblkid == 0);
|
|
|
|
|
if (off == 0 && len >= blksz) {
|
2013-12-09 22:37:51 +04:00
|
|
|
/*
|
|
|
|
|
* Freeing the whole block; fast-track this request.
|
|
|
|
|
*/
|
2008-12-03 23:09:06 +03:00
|
|
|
blkid = 0;
|
|
|
|
|
nblks = 1;
|
2019-07-08 23:18:50 +03:00
|
|
|
if (dn->dn_nlevels > 1) {
|
|
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_WRITER);
|
2016-10-31 20:42:37 +03:00
|
|
|
dnode_dirty_l1(dn, 0, tx);
|
2019-07-08 23:18:50 +03:00
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
|
|
|
|
}
|
2008-12-03 23:09:06 +03:00
|
|
|
goto done;
|
|
|
|
|
} else if (off >= blksz) {
|
|
|
|
|
/* Freeing past end-of-data */
|
2019-07-08 23:18:50 +03:00
|
|
|
return;
|
2008-11-20 23:01:55 +03:00
|
|
|
} else {
|
|
|
|
|
/* Freeing part of the block. */
|
|
|
|
|
head = blksz - off;
|
|
|
|
|
ASSERT3U(head, >, 0);
|
|
|
|
|
}
|
|
|
|
|
blkoff = off;
|
|
|
|
|
}
|
|
|
|
|
/* zero out any partial block data at the start of the range */
|
|
|
|
|
if (head) {
|
|
|
|
|
ASSERT3U(blkoff + head, ==, blksz);
|
|
|
|
|
if (len < head)
|
|
|
|
|
head = len;
|
2021-10-19 02:50:33 +03:00
|
|
|
dnode_partial_zero(dn, off, blkoff, head, tx);
|
2008-11-20 23:01:55 +03:00
|
|
|
off += head;
|
|
|
|
|
len -= head;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* If the range was less than one block, we're done */
|
2008-12-03 23:09:06 +03:00
|
|
|
if (len == 0)
|
2019-07-08 23:18:50 +03:00
|
|
|
return;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
/* If the remaining range is past end of file, we're done */
|
|
|
|
|
if ((off >> blkshift) > dn->dn_maxblkid)
|
2019-07-08 23:18:50 +03:00
|
|
|
return;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
ASSERT(ISP2(blksz));
|
|
|
|
|
if (trunc)
|
|
|
|
|
tail = 0;
|
|
|
|
|
else
|
|
|
|
|
tail = P2PHASE(len, blksz);
|
|
|
|
|
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(P2PHASE(off, blksz));
|
2008-12-03 23:09:06 +03:00
|
|
|
/* zero out any partial block data at the end of the range */
|
|
|
|
|
if (tail) {
|
|
|
|
|
if (len < tail)
|
|
|
|
|
tail = len;
|
2021-10-19 02:50:33 +03:00
|
|
|
dnode_partial_zero(dn, off + len, 0, tail, tx);
|
2008-12-03 23:09:06 +03:00
|
|
|
len -= tail;
|
|
|
|
|
}
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
/* If the range did not include a full block, we are done */
|
|
|
|
|
if (len == 0)
|
2019-07-08 23:18:50 +03:00
|
|
|
return;
|
2008-12-03 23:09:06 +03:00
|
|
|
|
|
|
|
|
ASSERT(IS_P2ALIGNED(off, blksz));
|
|
|
|
|
ASSERT(trunc || IS_P2ALIGNED(len, blksz));
|
|
|
|
|
blkid = off >> blkshift;
|
|
|
|
|
nblks = len >> blkshift;
|
|
|
|
|
if (trunc)
|
|
|
|
|
nblks += 1;
|
|
|
|
|
|
|
|
|
|
/*
|
2015-07-02 19:23:20 +03:00
|
|
|
* Dirty all the indirect blocks in this range. Note that only
|
|
|
|
|
* the first and last indirect blocks can actually be written
|
|
|
|
|
* (if they were partially freed) -- they must be dirtied, even if
|
|
|
|
|
* they do not exist on disk yet. The interior blocks will
|
|
|
|
|
* be freed by free_children(), so they will not actually be written.
|
|
|
|
|
* Even though these interior blocks will not be written, we
|
|
|
|
|
* dirty them for two reasons:
|
|
|
|
|
*
|
|
|
|
|
* - It ensures that the indirect blocks remain in memory until
|
|
|
|
|
* syncing context. (They have already been prefetched by
|
|
|
|
|
* dmu_tx_hold_free(), so we don't have to worry about reading
|
|
|
|
|
* them serially here.)
|
|
|
|
|
*
|
|
|
|
|
* - The dirty space accounting will put pressure on the txg sync
|
|
|
|
|
* mechanism to begin syncing, and to delay transactions if there
|
|
|
|
|
* is a large amount of freeing. Even though these indirect
|
|
|
|
|
* blocks will not be written, we could need to write the same
|
|
|
|
|
* amount of space if we copy the freed BPs into deadlists.
|
2008-12-03 23:09:06 +03:00
|
|
|
*/
|
|
|
|
|
if (dn->dn_nlevels > 1) {
|
2019-07-08 23:18:50 +03:00
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_WRITER);
|
2017-11-04 23:25:13 +03:00
|
|
|
uint64_t first, last;
|
2008-12-03 23:09:06 +03:00
|
|
|
|
|
|
|
|
first = blkid >> epbs;
|
2015-07-02 19:23:20 +03:00
|
|
|
dnode_dirty_l1(dn, first, tx);
|
2008-12-03 23:09:06 +03:00
|
|
|
if (trunc)
|
|
|
|
|
last = dn->dn_maxblkid >> epbs;
|
|
|
|
|
else
|
|
|
|
|
last = (blkid + nblks - 1) >> epbs;
|
2015-07-02 19:23:20 +03:00
|
|
|
if (last != first)
|
|
|
|
|
dnode_dirty_l1(dn, last, tx);
|
|
|
|
|
|
OpenZFS 9438 - Holes can lose birth time info if a block has a mix of birth times
As reported by https://github.com/zfsonlinux/zfs/issues/4996, there is
yet another hole birth issue. In this one, if a block is entirely holes,
but the birth times are not all the same, we lose that information by
creating one hole with the current txg as its birth time.
The ZoL PR's fix approach is incorrect. Ultimately, the problem here is
that when you truncate and write a file in the same transaction group,
the dbuf for the indirect block will be zeroed out to deal with the
truncation, and then written for the write. During this process, we will
lose hole birth time information for any holes in the range. In the case
where a dnode is being freed, we need to determine whether the block
should be converted to a higher-level hole in the zio pipeline, and if
so do it when the dnode is being synced out.
Porting Notes:
* The DMU_OBJECT_END change in zfs_znode.c was already applied.
* Added test cases from #5675 provided by @rincebrain for hole_birth
issues. These test cases should be pushed upstream to OpenZFS.
* Updated mk_files which is used by several rsend tests so the
files created are a little more interesting and may contain holes.
Authored by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>
OpenZFS-issue: https://www.illumos.org/issues/9438
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/738e2a3c
External-issue: DLPX-46861
Closes #7746
2016-09-20 20:02:29 +03:00
|
|
|
dnode_dirty_l1range(dn, first, last, tx);
|
|
|
|
|
|
2017-11-04 23:25:13 +03:00
|
|
|
int shift = dn->dn_datablkshift + dn->dn_indblkshift -
|
2015-07-02 19:23:20 +03:00
|
|
|
SPA_BLKPTRSHIFT;
|
2017-11-04 23:25:13 +03:00
|
|
|
for (uint64_t i = first + 1; i < last; i++) {
|
2015-07-02 19:23:20 +03:00
|
|
|
/*
|
|
|
|
|
* Set i to the blockid of the next non-hole
|
|
|
|
|
* level-1 indirect block at or after i. Note
|
|
|
|
|
* that dnode_next_offset() operates in terms of
|
|
|
|
|
* level-0-equivalent bytes.
|
|
|
|
|
*/
|
2017-11-04 23:25:13 +03:00
|
|
|
uint64_t ibyte = i << shift;
|
|
|
|
|
int err = dnode_next_offset(dn, DNODE_FIND_HAVELOCK,
|
2015-07-02 19:23:20 +03:00
|
|
|
&ibyte, 2, 1, 0);
|
|
|
|
|
i = ibyte >> shift;
|
|
|
|
|
if (i >= last)
|
|
|
|
|
break;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Normally we should not see an error, either
|
|
|
|
|
* from dnode_next_offset() or dbuf_hold_level()
|
|
|
|
|
* (except for ESRCH from dnode_next_offset).
|
|
|
|
|
* If there is an i/o error, then when we read
|
|
|
|
|
* this block in syncing context, it will use
|
|
|
|
|
* ZIO_FLAG_MUSTSUCCEED, and thus hang/panic according
|
|
|
|
|
* to the "failmode" property. dnode_next_offset()
|
|
|
|
|
* doesn't have a flag to indicate MUSTSUCCEED.
|
|
|
|
|
*/
|
|
|
|
|
if (err != 0)
|
|
|
|
|
break;
|
|
|
|
|
|
|
|
|
|
dnode_dirty_l1(dn, i, tx);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
2019-07-08 23:18:50 +03:00
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
2013-12-09 22:37:51 +04:00
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
done:
|
|
|
|
|
/*
|
|
|
|
|
* Add this range to the dnode range list.
|
|
|
|
|
* We will finish up this free operation in the syncing phase.
|
|
|
|
|
*/
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_enter(&dn->dn_mtx);
|
|
|
|
|
{
|
2020-08-20 21:45:20 +03:00
|
|
|
int txgoff = tx->tx_txg & TXG_MASK;
|
|
|
|
|
if (dn->dn_free_ranges[txgoff] == NULL) {
|
|
|
|
|
dn->dn_free_ranges[txgoff] = range_tree_create(NULL,
|
|
|
|
|
RANGE_SEG64, NULL, 0, 0);
|
|
|
|
|
}
|
|
|
|
|
range_tree_clear(dn->dn_free_ranges[txgoff], blkid, nblks);
|
|
|
|
|
range_tree_add(dn->dn_free_ranges[txgoff], blkid, nblks);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
2014-04-16 07:40:22 +04:00
|
|
|
dprintf_dnode(dn, "blkid=%llu nblks=%llu txg=%llu\n",
|
2021-06-23 07:53:45 +03:00
|
|
|
(u_longlong_t)blkid, (u_longlong_t)nblks,
|
|
|
|
|
(u_longlong_t)tx->tx_txg);
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
dbuf_free_range(dn, blkid, blkid + nblks - 1, tx);
|
2008-11-20 23:01:55 +03:00
|
|
|
dnode_setdirty(dn, tx);
|
|
|
|
|
}
|
|
|
|
|
|
2010-05-29 00:45:14 +04:00
|
|
|
static boolean_t
|
|
|
|
|
dnode_spill_freed(dnode_t *dn)
|
|
|
|
|
{
|
|
|
|
|
int i;
|
|
|
|
|
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
|
|
|
|
for (i = 0; i < TXG_SIZE; i++) {
|
|
|
|
|
if (dn->dn_rm_spillblk[i] == DN_KILL_SPILLBLK)
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
return (i < TXG_SIZE);
|
|
|
|
|
}
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
/* return TRUE if this blkid was freed in a recent txg, or FALSE if it wasn't */
|
|
|
|
|
uint64_t
|
|
|
|
|
dnode_block_freed(dnode_t *dn, uint64_t blkid)
|
|
|
|
|
{
|
|
|
|
|
int i;
|
|
|
|
|
|
2010-05-29 00:45:14 +04:00
|
|
|
if (blkid == DMU_BONUS_BLKID)
|
2008-11-20 23:01:55 +03:00
|
|
|
return (FALSE);
|
|
|
|
|
|
|
|
|
|
if (dn->dn_free_txg)
|
|
|
|
|
return (TRUE);
|
|
|
|
|
|
2010-05-29 00:45:14 +04:00
|
|
|
if (blkid == DMU_SPILL_BLKID)
|
|
|
|
|
return (dnode_spill_freed(dn));
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
mutex_enter(&dn->dn_mtx);
|
|
|
|
|
for (i = 0; i < TXG_SIZE; i++) {
|
2014-04-16 07:40:22 +04:00
|
|
|
if (dn->dn_free_ranges[i] != NULL &&
|
|
|
|
|
range_tree_contains(dn->dn_free_ranges[i], blkid, 1))
|
2008-11-20 23:01:55 +03:00
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
return (i < TXG_SIZE);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* call from syncing context when we actually write/free space for this dnode */
|
|
|
|
|
void
|
|
|
|
|
dnode_diduse_space(dnode_t *dn, int64_t delta)
|
|
|
|
|
{
|
|
|
|
|
uint64_t space;
|
|
|
|
|
dprintf_dnode(dn, "dn=%p dnp=%p used=%llu delta=%lld\n",
|
|
|
|
|
dn, dn->dn_phys,
|
|
|
|
|
(u_longlong_t)dn->dn_phys->dn_used,
|
|
|
|
|
(longlong_t)delta);
|
|
|
|
|
|
|
|
|
|
mutex_enter(&dn->dn_mtx);
|
|
|
|
|
space = DN_USED_BYTES(dn->dn_phys);
|
|
|
|
|
if (delta > 0) {
|
|
|
|
|
ASSERT3U(space + delta, >=, space); /* no overflow */
|
|
|
|
|
} else {
|
|
|
|
|
ASSERT3U(space, >=, -delta); /* no underflow */
|
|
|
|
|
}
|
|
|
|
|
space += delta;
|
|
|
|
|
if (spa_version(dn->dn_objset->os_spa) < SPA_VERSION_DNODE_BYTES) {
|
|
|
|
|
ASSERT((dn->dn_phys->dn_flags & DNODE_FLAG_USED_BYTES) == 0);
|
2013-05-11 01:17:03 +04:00
|
|
|
ASSERT0(P2PHASE(space, 1<<DEV_BSHIFT));
|
2008-11-20 23:01:55 +03:00
|
|
|
dn->dn_phys->dn_used = space >> DEV_BSHIFT;
|
|
|
|
|
} else {
|
|
|
|
|
dn->dn_phys->dn_used = space;
|
|
|
|
|
dn->dn_phys->dn_flags |= DNODE_FLAG_USED_BYTES;
|
|
|
|
|
}
|
|
|
|
|
mutex_exit(&dn->dn_mtx);
|
|
|
|
|
}
|
|
|
|
|
|
2009-08-18 22:43:27 +04:00
|
|
|
/*
|
2013-06-11 21:12:34 +04:00
|
|
|
* Scans a block at the indicated "level" looking for a hole or data,
|
|
|
|
|
* depending on 'flags'.
|
|
|
|
|
*
|
|
|
|
|
* If level > 0, then we are scanning an indirect block looking at its
|
|
|
|
|
* pointers. If level == 0, then we are looking at a block of dnodes.
|
|
|
|
|
*
|
|
|
|
|
* If we don't find what we are looking for in the block, we return ESRCH.
|
|
|
|
|
* Otherwise, return with *offset pointing to the beginning (if searching
|
|
|
|
|
* forwards) or end (if searching backwards) of the range covered by the
|
|
|
|
|
* block pointer we matched on (or dnode).
|
2009-08-18 22:43:27 +04:00
|
|
|
*
|
|
|
|
|
* The basic search algorithm used below by dnode_next_offset() is to
|
|
|
|
|
* use this function to search up the block tree (widen the search) until
|
|
|
|
|
* we find something (i.e., we don't return ESRCH) and then search back
|
|
|
|
|
* down the tree (narrow the search) until we reach our original search
|
|
|
|
|
* level.
|
|
|
|
|
*/
|
2008-11-20 23:01:55 +03:00
|
|
|
static int
|
2008-12-03 23:09:06 +03:00
|
|
|
dnode_next_offset_level(dnode_t *dn, int flags, uint64_t *offset,
|
2015-12-22 04:31:57 +03:00
|
|
|
int lvl, uint64_t blkfill, uint64_t txg)
|
2008-11-20 23:01:55 +03:00
|
|
|
{
|
|
|
|
|
dmu_buf_impl_t *db = NULL;
|
|
|
|
|
void *data = NULL;
|
|
|
|
|
uint64_t epbs = dn->dn_phys->dn_indblkshift - SPA_BLKPTRSHIFT;
|
|
|
|
|
uint64_t epb = 1ULL << epbs;
|
|
|
|
|
uint64_t minfill, maxfill;
|
2008-12-03 23:09:06 +03:00
|
|
|
boolean_t hole;
|
|
|
|
|
int i, inc, error, span;
|
2008-11-20 23:01:55 +03:00
|
|
|
|
2019-07-08 23:18:50 +03:00
|
|
|
ASSERT(RW_LOCK_HELD(&dn->dn_struct_rwlock));
|
|
|
|
|
|
2009-07-03 02:44:48 +04:00
|
|
|
hole = ((flags & DNODE_FIND_HOLE) != 0);
|
2008-12-03 23:09:06 +03:00
|
|
|
inc = (flags & DNODE_FIND_BACKWARDS) ? -1 : 1;
|
|
|
|
|
ASSERT(txg == 0 || !hole);
|
|
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
if (lvl == dn->dn_phys->dn_nlevels) {
|
|
|
|
|
error = 0;
|
|
|
|
|
epb = dn->dn_phys->dn_nblkptr;
|
|
|
|
|
data = dn->dn_phys->dn_blkptr;
|
|
|
|
|
} else {
|
2015-12-22 04:31:57 +03:00
|
|
|
uint64_t blkid = dbuf_whichblock(dn, lvl, *offset);
|
|
|
|
|
error = dbuf_hold_impl(dn, lvl, blkid, TRUE, FALSE, FTAG, &db);
|
2008-11-20 23:01:55 +03:00
|
|
|
if (error) {
|
2008-12-03 23:09:06 +03:00
|
|
|
if (error != ENOENT)
|
|
|
|
|
return (error);
|
|
|
|
|
if (hole)
|
|
|
|
|
return (0);
|
|
|
|
|
/*
|
|
|
|
|
* This can only happen when we are searching up
|
|
|
|
|
* the block tree for data. We don't really need to
|
|
|
|
|
* adjust the offset, as we will just end up looking
|
|
|
|
|
* at the pointer to this block in its parent, and its
|
|
|
|
|
* going to be unallocated, so we will skip over it.
|
|
|
|
|
*/
|
2013-03-08 22:41:28 +04:00
|
|
|
return (SET_ERROR(ESRCH));
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
Native Encryption for ZFS on Linux
This change incorporates three major pieces:
The first change is a keystore that manages wrapping
and encryption keys for encrypted datasets. These
commands mostly involve manipulating the new
DSL Crypto Key ZAP Objects that live in the MOS. Each
encrypted dataset has its own DSL Crypto Key that is
protected with a user's key. This level of indirection
allows users to change their keys without re-encrypting
their entire datasets. The change implements the new
subcommands "zfs load-key", "zfs unload-key" and
"zfs change-key" which allow the user to manage their
encryption keys and settings. In addition, several new
flags and properties have been added to allow dataset
creation and to make mounting and unmounting more
convenient.
The second piece of this patch provides the ability to
encrypt, decyrpt, and authenticate protected datasets.
Each object set maintains a Merkel tree of Message
Authentication Codes that protect the lower layers,
similarly to how checksums are maintained. This part
impacts the zio layer, which handles the actual
encryption and generation of MACs, as well as the ARC
and DMU, which need to be able to handle encrypted
buffers and protected data.
The last addition is the ability to do raw, encrypted
sends and receives. The idea here is to send raw
encrypted and compressed data and receive it exactly
as is on a backup system. This means that the dataset
on the receiving system is protected using the same
user key that is in use on the sending side. By doing
so, datasets can be efficiently backed up to an
untrusted system without fear of data being
compromised.
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Jorgen Lundman <lundman@lundman.net>
Signed-off-by: Tom Caputi <tcaputi@datto.com>
Closes #494
Closes #5769
2017-08-14 20:36:48 +03:00
|
|
|
error = dbuf_read(db, NULL,
|
2020-09-25 23:49:22 +03:00
|
|
|
DB_RF_CANFAIL | DB_RF_HAVESTRUCT |
|
|
|
|
|
DB_RF_NO_DECRYPT | DB_RF_NOPREFETCH);
|
2008-11-20 23:01:55 +03:00
|
|
|
if (error) {
|
|
|
|
|
dbuf_rele(db, FTAG);
|
|
|
|
|
return (error);
|
|
|
|
|
}
|
|
|
|
|
data = db->db.db_data;
|
2019-07-08 23:18:50 +03:00
|
|
|
rw_enter(&db->db_rwlock, RW_READER);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
2013-12-09 22:37:51 +04:00
|
|
|
if (db != NULL && txg != 0 && (db->db_blkptr == NULL ||
|
|
|
|
|
db->db_blkptr->blk_birth <= txg ||
|
|
|
|
|
BP_IS_HOLE(db->db_blkptr))) {
|
2008-12-03 23:09:06 +03:00
|
|
|
/*
|
|
|
|
|
* This can only happen when we are searching up the tree
|
|
|
|
|
* and these conditions mean that we need to keep climbing.
|
|
|
|
|
*/
|
2013-03-08 22:41:28 +04:00
|
|
|
error = SET_ERROR(ESRCH);
|
2008-11-20 23:01:55 +03:00
|
|
|
} else if (lvl == 0) {
|
|
|
|
|
dnode_phys_t *dnp = data;
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
|
2008-11-20 23:01:55 +03:00
|
|
|
ASSERT(dn->dn_type == DMU_OT_DNODE);
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
ASSERT(!(flags & DNODE_FIND_BACKWARDS));
|
2008-11-20 23:01:55 +03:00
|
|
|
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
for (i = (*offset >> DNODE_SHIFT) & (blkfill - 1);
|
|
|
|
|
i < blkfill; i += dnp[i].dn_extra_slots + 1) {
|
2009-07-03 02:44:48 +04:00
|
|
|
if ((dnp[i].dn_type == DMU_OT_NONE) == hole)
|
2008-11-20 23:01:55 +03:00
|
|
|
break;
|
|
|
|
|
}
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
|
|
|
|
|
if (i == blkfill)
|
2013-03-08 22:41:28 +04:00
|
|
|
error = SET_ERROR(ESRCH);
|
Implement large_dnode pool feature
Justification
-------------
This feature adds support for variable length dnodes. Our motivation is
to eliminate the overhead associated with using spill blocks. Spill
blocks are used to store system attribute data (i.e. file metadata) that
does not fit in the dnode's bonus buffer. By allowing a larger bonus
buffer area the use of a spill block can be avoided. Spill blocks
potentially incur an additional read I/O for every dnode in a dnode
block. As a worst case example, reading 32 dnodes from a 16k dnode block
and all of the spill blocks could issue 33 separate reads. Now suppose
those dnodes have size 1024 and therefore don't need spill blocks. Then
the worst case number of blocks read is reduced to from 33 to two--one
per dnode block. In practice spill blocks may tend to be co-located on
disk with the dnode blocks so the reduction in I/O would not be this
drastic. In a badly fragmented pool, however, the improvement could be
significant.
ZFS-on-Linux systems that make heavy use of extended attributes would
benefit from this feature. In particular, ZFS-on-Linux supports the
xattr=sa dataset property which allows file extended attribute data
to be stored in the dnode bonus buffer as an alternative to the
traditional directory-based format. Workloads such as SELinux and the
Lustre distributed filesystem often store enough xattr data to force
spill bocks when xattr=sa is in effect. Large dnodes may therefore
provide a performance benefit to such systems.
Other use cases that may benefit from this feature include files with
large ACLs and symbolic links with long target names. Furthermore,
this feature may be desirable on other platforms in case future
applications or features are developed that could make use of a
larger bonus buffer area.
Implementation
--------------
The size of a dnode may be a multiple of 512 bytes up to the size of
a dnode block (currently 16384 bytes). A dn_extra_slots field was
added to the current on-disk dnode_phys_t structure to describe the
size of the physical dnode on disk. The 8 bits for this field were
taken from the zero filled dn_pad2 field. The field represents how
many "extra" dnode_phys_t slots a dnode consumes in its dnode block.
This convention results in a value of 0 for 512 byte dnodes which
preserves on-disk format compatibility with older software.
Similarly, the in-memory dnode_t structure has a new dn_num_slots field
to represent the total number of dnode_phys_t slots consumed on disk.
Thus dn->dn_num_slots is 1 greater than the corresponding
dnp->dn_extra_slots. This difference in convention was adopted
because, unlike on-disk structures, backward compatibility is not a
concern for in-memory objects, so we used a more natural way to
represent size for a dnode_t.
The default size for newly created dnodes is determined by the value of
a new "dnodesize" dataset property. By default the property is set to
"legacy" which is compatible with older software. Setting the property
to "auto" will allow the filesystem to choose the most suitable dnode
size. Currently this just sets the default dnode size to 1k, but future
code improvements could dynamically choose a size based on observed
workload patterns. Dnodes of varying sizes can coexist within the same
dataset and even within the same dnode block. For example, to enable
automatically-sized dnodes, run
# zfs set dnodesize=auto tank/fish
The user can also specify literal values for the dnodesize property.
These are currently limited to powers of two from 1k to 16k. The
power-of-2 limitation is only for simplicity of the user interface.
Internally the implementation can handle any multiple of 512 up to 16k,
and consumers of the DMU API can specify any legal dnode value.
The size of a new dnode is determined at object allocation time and
stored as a new field in the znode in-memory structure. New DMU
interfaces are added to allow the consumer to specify the dnode size
that a newly allocated object should use. Existing interfaces are
unchanged to avoid having to update every call site and to preserve
compatibility with external consumers such as Lustre. The new
interfaces names are given below. The versions of these functions that
don't take a dnodesize parameter now just call the _dnsize() versions
with a dnodesize of 0, which means use the legacy dnode size.
New DMU interfaces:
dmu_object_alloc_dnsize()
dmu_object_claim_dnsize()
dmu_object_reclaim_dnsize()
New ZAP interfaces:
zap_create_dnsize()
zap_create_norm_dnsize()
zap_create_flags_dnsize()
zap_create_claim_norm_dnsize()
zap_create_link_dnsize()
The constant DN_MAX_BONUSLEN is renamed to DN_OLD_MAX_BONUSLEN. The
spa_maxdnodesize() function should be used to determine the maximum
bonus length for a pool.
These are a few noteworthy changes to key functions:
* The prototype for dnode_hold_impl() now takes a "slots" parameter.
When the DNODE_MUST_BE_FREE flag is set, this parameter is used to
ensure the hole at the specified object offset is large enough to
hold the dnode being created. The slots parameter is also used
to ensure a dnode does not span multiple dnode blocks. In both of
these cases, if a failure occurs, ENOSPC is returned. Keep in mind,
these failure cases are only possible when using DNODE_MUST_BE_FREE.
If the DNODE_MUST_BE_ALLOCATED flag is set, "slots" must be 0.
dnode_hold_impl() will check if the requested dnode is already
consumed as an extra dnode slot by an large dnode, in which case
it returns ENOENT.
* The function dmu_object_alloc() advances to the next dnode block
if dnode_hold_impl() returns an error for a requested object.
This is because the beginning of the next dnode block is the only
location it can safely assume to either be a hole or a valid
starting point for a dnode.
* dnode_next_offset_level() and other functions that iterate
through dnode blocks may no longer use a simple array indexing
scheme. These now use the current dnode's dn_num_slots field to
advance to the next dnode in the block. This is to ensure we
properly skip the current dnode's bonus area and don't interpret it
as a valid dnode.
zdb
---
The zdb command was updated to display a dnode's size under the
"dnsize" column when the object is dumped.
For ZIL create log records, zdb will now display the slot count for
the object.
ztest
-----
Ztest chooses a random dnodesize for every newly created object. The
random distribution is more heavily weighted toward small dnodes to
better simulate real-world datasets.
Unused bonus buffer space is filled with non-zero values computed from
the object number, dataset id, offset, and generation number. This
helps ensure that the dnode traversal code properly skips the interior
regions of large dnodes, and that these interior regions are not
overwritten by data belonging to other dnodes. A new test visits each
object in a dataset. It verifies that the actual dnode size matches what
was stored in the ztest block tag when it was created. It also verifies
that the unused bonus buffer space is filled with the expected data
patterns.
ZFS Test Suite
--------------
Added six new large dnode-specific tests, and integrated the dnodesize
property into existing tests for zfs allow and send/recv.
Send/Receive
------------
ZFS send streams for datasets containing large dnodes cannot be received
on pools that don't support the large_dnode feature. A send stream with
large dnodes sets a DMU_BACKUP_FEATURE_LARGE_DNODE flag which will be
unrecognized by an incompatible receiving pool so that the zfs receive
will fail gracefully.
While not implemented here, it may be possible to generate a
backward-compatible send stream from a dataset containing large
dnodes. The implementation may be tricky, however, because the send
object record for a large dnode would need to be resized to a 512
byte dnode, possibly kicking in a spill block in the process. This
means we would need to construct a new SA layout and possibly
register it in the SA layout object. The SA layout is normally just
sent as an ordinary object record. But if we are constructing new
layouts while generating the send stream we'd have to build the SA
layout object dynamically and send it at the end of the stream.
For sending and receiving between pools that do support large dnodes,
the drr_object send record type is extended with a new field to store
the dnode slot count. This field was repurposed from unused padding
in the structure.
ZIL Replay
----------
The dnode slot count is stored in the uppermost 8 bits of the lr_foid
field. The bits were unused as the object id is currently capped at
48 bits.
Resizing Dnodes
---------------
It should be possible to resize a dnode when it is dirtied if the
current dnodesize dataset property differs from the dnode's size, but
this functionality is not currently implemented. Clearly a dnode can
only grow if there are sufficient contiguous unused slots in the
dnode block, but it should always be possible to shrink a dnode.
Growing dnodes may be useful to reduce fragmentation in a pool with
many spill blocks in use. Shrinking dnodes may be useful to allow
sending a dataset to a pool that doesn't support the large_dnode
feature.
Feature Reference Counting
--------------------------
The reference count for the large_dnode pool feature tracks the
number of datasets that have ever contained a dnode of size larger
than 512 bytes. The first time a large dnode is created in a dataset
the dataset is converted to an extensible dataset. This is a one-way
operation and the only way to decrement the feature count is to
destroy the dataset, even if the dataset no longer contains any large
dnodes. The complexity of reference counting on a per-dnode basis was
too high, so we chose to track it on a per-dataset basis similarly to
the large_block feature.
Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #3542
2016-03-17 04:25:34 +03:00
|
|
|
|
|
|
|
|
*offset = (*offset & ~(DNODE_BLOCK_SIZE - 1)) +
|
|
|
|
|
(i << DNODE_SHIFT);
|
2008-11-20 23:01:55 +03:00
|
|
|
} else {
|
|
|
|
|
blkptr_t *bp = data;
|
2009-08-18 22:43:27 +04:00
|
|
|
uint64_t start = *offset;
|
2008-11-20 23:01:55 +03:00
|
|
|
span = (lvl - 1) * epbs + dn->dn_datablkshift;
|
|
|
|
|
minfill = 0;
|
|
|
|
|
maxfill = blkfill << ((lvl - 1) * epbs);
|
|
|
|
|
|
|
|
|
|
if (hole)
|
|
|
|
|
maxfill--;
|
|
|
|
|
else
|
|
|
|
|
minfill++;
|
|
|
|
|
|
2016-08-31 11:12:08 +03:00
|
|
|
if (span >= 8 * sizeof (*offset)) {
|
|
|
|
|
/* This only happens on the highest indirection level */
|
|
|
|
|
ASSERT3U((lvl - 1), ==, dn->dn_phys->dn_nlevels - 1);
|
|
|
|
|
*offset = 0;
|
|
|
|
|
} else {
|
|
|
|
|
*offset = *offset >> span;
|
|
|
|
|
}
|
|
|
|
|
|
2009-08-18 22:43:27 +04:00
|
|
|
for (i = BF64_GET(*offset, 0, epbs);
|
2008-12-03 23:09:06 +03:00
|
|
|
i >= 0 && i < epb; i += inc) {
|
2014-06-06 01:19:08 +04:00
|
|
|
if (BP_GET_FILL(&bp[i]) >= minfill &&
|
|
|
|
|
BP_GET_FILL(&bp[i]) <= maxfill &&
|
2008-12-03 23:09:06 +03:00
|
|
|
(hole || bp[i].blk_birth > txg))
|
2008-11-20 23:01:55 +03:00
|
|
|
break;
|
2009-08-18 22:43:27 +04:00
|
|
|
if (inc > 0 || *offset > 0)
|
|
|
|
|
*offset += inc;
|
|
|
|
|
}
|
2016-08-31 11:12:08 +03:00
|
|
|
|
|
|
|
|
if (span >= 8 * sizeof (*offset)) {
|
|
|
|
|
*offset = start;
|
|
|
|
|
} else {
|
|
|
|
|
*offset = *offset << span;
|
|
|
|
|
}
|
|
|
|
|
|
2009-08-18 22:43:27 +04:00
|
|
|
if (inc < 0) {
|
|
|
|
|
/* traversing backwards; position offset at the end */
|
2023-03-17 00:27:49 +03:00
|
|
|
if (span < 8 * sizeof (*offset))
|
|
|
|
|
*offset = MIN(*offset + (1ULL << span) - 1,
|
|
|
|
|
start);
|
2009-08-18 22:43:27 +04:00
|
|
|
} else if (*offset < start) {
|
|
|
|
|
*offset = start;
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
2009-08-18 22:43:27 +04:00
|
|
|
if (i < 0 || i >= epb)
|
2013-03-08 22:41:28 +04:00
|
|
|
error = SET_ERROR(ESRCH);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
2019-07-08 23:18:50 +03:00
|
|
|
if (db != NULL) {
|
|
|
|
|
rw_exit(&db->db_rwlock);
|
2008-11-20 23:01:55 +03:00
|
|
|
dbuf_rele(db, FTAG);
|
2019-07-08 23:18:50 +03:00
|
|
|
}
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
return (error);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Find the next hole, data, or sparse region at or after *offset.
|
|
|
|
|
* The value 'blkfill' tells us how many items we expect to find
|
|
|
|
|
* in an L0 data block; this value is 1 for normal objects,
|
|
|
|
|
* DNODES_PER_BLOCK for the meta dnode, and some fraction of
|
|
|
|
|
* DNODES_PER_BLOCK when searching for sparse regions thereof.
|
|
|
|
|
*
|
|
|
|
|
* Examples:
|
|
|
|
|
*
|
2008-12-03 23:09:06 +03:00
|
|
|
* dnode_next_offset(dn, flags, offset, 1, 1, 0);
|
|
|
|
|
* Finds the next/previous hole/data in a file.
|
2008-11-20 23:01:55 +03:00
|
|
|
* Used in dmu_offset_next().
|
|
|
|
|
*
|
2008-12-03 23:09:06 +03:00
|
|
|
* dnode_next_offset(mdn, flags, offset, 0, DNODES_PER_BLOCK, txg);
|
2008-11-20 23:01:55 +03:00
|
|
|
* Finds the next free/allocated dnode an objset's meta-dnode.
|
|
|
|
|
* Only finds objects that have new contents since txg (ie.
|
|
|
|
|
* bonus buffer changes and content removal are ignored).
|
|
|
|
|
* Used in dmu_object_next().
|
|
|
|
|
*
|
2008-12-03 23:09:06 +03:00
|
|
|
* dnode_next_offset(mdn, DNODE_FIND_HOLE, offset, 2, DNODES_PER_BLOCK >> 2, 0);
|
2008-11-20 23:01:55 +03:00
|
|
|
* Finds the next L2 meta-dnode bp that's at most 1/4 full.
|
|
|
|
|
* Used in dmu_object_alloc().
|
|
|
|
|
*/
|
|
|
|
|
int
|
2008-12-03 23:09:06 +03:00
|
|
|
dnode_next_offset(dnode_t *dn, int flags, uint64_t *offset,
|
2008-11-20 23:01:55 +03:00
|
|
|
int minlvl, uint64_t blkfill, uint64_t txg)
|
|
|
|
|
{
|
2008-12-03 23:09:06 +03:00
|
|
|
uint64_t initial_offset = *offset;
|
2008-11-20 23:01:55 +03:00
|
|
|
int lvl, maxlvl;
|
|
|
|
|
int error = 0;
|
|
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
if (!(flags & DNODE_FIND_HAVELOCK))
|
|
|
|
|
rw_enter(&dn->dn_struct_rwlock, RW_READER);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
if (dn->dn_phys->dn_nlevels == 0) {
|
2023-04-18 18:41:52 +03:00
|
|
|
error = SET_ERROR(ESRCH);
|
2008-12-03 23:09:06 +03:00
|
|
|
goto out;
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (dn->dn_datablkshift == 0) {
|
|
|
|
|
if (*offset < dn->dn_datablksz) {
|
2008-12-03 23:09:06 +03:00
|
|
|
if (flags & DNODE_FIND_HOLE)
|
2008-11-20 23:01:55 +03:00
|
|
|
*offset = dn->dn_datablksz;
|
|
|
|
|
} else {
|
2013-03-08 22:41:28 +04:00
|
|
|
error = SET_ERROR(ESRCH);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
2008-12-03 23:09:06 +03:00
|
|
|
goto out;
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
maxlvl = dn->dn_phys->dn_nlevels;
|
|
|
|
|
|
|
|
|
|
for (lvl = minlvl; lvl <= maxlvl; lvl++) {
|
|
|
|
|
error = dnode_next_offset_level(dn,
|
2008-12-03 23:09:06 +03:00
|
|
|
flags, offset, lvl, blkfill, txg);
|
2008-11-20 23:01:55 +03:00
|
|
|
if (error != ESRCH)
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
while (error == 0 && --lvl >= minlvl) {
|
2008-11-20 23:01:55 +03:00
|
|
|
error = dnode_next_offset_level(dn,
|
2008-12-03 23:09:06 +03:00
|
|
|
flags, offset, lvl, blkfill, txg);
|
2008-11-20 23:01:55 +03:00
|
|
|
}
|
|
|
|
|
|
2014-09-17 19:25:10 +04:00
|
|
|
/*
|
|
|
|
|
* There's always a "virtual hole" at the end of the object, even
|
|
|
|
|
* if all BP's which physically exist are non-holes.
|
|
|
|
|
*/
|
|
|
|
|
if ((flags & DNODE_FIND_HOLE) && error == ESRCH && txg == 0 &&
|
|
|
|
|
minlvl == 1 && blkfill == 1 && !(flags & DNODE_FIND_BACKWARDS)) {
|
|
|
|
|
error = 0;
|
|
|
|
|
}
|
|
|
|
|
|
2008-12-03 23:09:06 +03:00
|
|
|
if (error == 0 && (flags & DNODE_FIND_BACKWARDS ?
|
|
|
|
|
initial_offset < *offset : initial_offset > *offset))
|
2013-03-08 22:41:28 +04:00
|
|
|
error = SET_ERROR(ESRCH);
|
2008-12-03 23:09:06 +03:00
|
|
|
out:
|
|
|
|
|
if (!(flags & DNODE_FIND_HAVELOCK))
|
|
|
|
|
rw_exit(&dn->dn_struct_rwlock);
|
2008-11-20 23:01:55 +03:00
|
|
|
|
|
|
|
|
return (error);
|
|
|
|
|
}
|
2019-07-16 02:11:55 +03:00
|
|
|
|
|
|
|
|
#if defined(_KERNEL)
|
|
|
|
|
EXPORT_SYMBOL(dnode_hold);
|
|
|
|
|
EXPORT_SYMBOL(dnode_rele);
|
|
|
|
|
EXPORT_SYMBOL(dnode_set_nlevels);
|
|
|
|
|
EXPORT_SYMBOL(dnode_set_blksz);
|
|
|
|
|
EXPORT_SYMBOL(dnode_free_range);
|
|
|
|
|
EXPORT_SYMBOL(dnode_evict_dbufs);
|
|
|
|
|
EXPORT_SYMBOL(dnode_evict_bonus);
|
|
|
|
|
#endif
|
2023-01-11 20:38:20 +03:00
|
|
|
|
|
|
|
|
ZFS_MODULE_PARAM(zfs, zfs_, default_bs, INT, ZMOD_RW,
|
|
|
|
|
"Default dnode block shift");
|
|
|
|
|
ZFS_MODULE_PARAM(zfs, zfs_, default_ibs, INT, ZMOD_RW,
|
|
|
|
|
"Default dnode indirect block shift");
|