mirror of
https://git.proxmox.com/git/mirror_zfs.git
synced 2024-12-26 11:19:32 +03:00
196bee4cfd
Deduplicated send streams (i.e. `zfs send -D` and `zfs receive` of such streams) are deprecated. Deduplicated send streams can be received by first converting them to non-deduplicated with the `zstream redup` command. This commit removes the code for sending and receiving deduplicated send streams. `zfs send -D` will now print a warning, ignore the `-D` flag, and generate a regular (non-deduplicated) send stream. `zfs receive` of a deduplicated send stream will print an error message and fail. The resulting code simplification (especially in the kernel's support for receiving dedup streams) should help enable future performance enhancements. Several new tests are added which leverage `zstream redup`. Reviewed-by: Paul Dagnelie <pcd@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Matthew Ahrens <mahrens@delphix.com> Issue #7887 Issue #10117 Issue #10156 Closes #10212
376 lines
13 KiB
Groff
376 lines
13 KiB
Groff
.\"
|
||
.\" CDDL HEADER START
|
||
.\"
|
||
.\" The contents of this file are subject to the terms of the
|
||
.\" Common Development and Distribution License (the "License").
|
||
.\" You may not use this file except in compliance with the License.
|
||
.\"
|
||
.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
|
||
.\" or http://www.opensolaris.org/os/licensing.
|
||
.\" See the License for the specific language governing permissions
|
||
.\" and limitations under the License.
|
||
.\"
|
||
.\" When distributing Covered Code, include this CDDL HEADER in each
|
||
.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
|
||
.\" If applicable, add the following below this CDDL HEADER, with the
|
||
.\" fields enclosed by brackets "[]" replaced with your own identifying
|
||
.\" information: Portions Copyright [yyyy] [name of copyright owner]
|
||
.\"
|
||
.\" CDDL HEADER END
|
||
.\"
|
||
.\"
|
||
.\" Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved.
|
||
.\" Copyright 2011 Joshua M. Clulow <josh@sysmgr.org>
|
||
.\" Copyright (c) 2011, 2019 by Delphix. All rights reserved.
|
||
.\" Copyright (c) 2013 by Saso Kiselkov. All rights reserved.
|
||
.\" Copyright (c) 2014, Joyent, Inc. All rights reserved.
|
||
.\" Copyright (c) 2014 by Adam Stevko. All rights reserved.
|
||
.\" Copyright (c) 2014 Integros [integros.com]
|
||
.\" Copyright 2019 Richard Laager. All rights reserved.
|
||
.\" Copyright 2018 Nexenta Systems, Inc.
|
||
.\" Copyright 2019 Joyent, Inc.
|
||
.\"
|
||
.Dd February 16, 2020
|
||
.Dt ZFS-RECEIVE 8
|
||
.Os Linux
|
||
.Sh NAME
|
||
.Nm zfs Ns Pf - Cm receive
|
||
.Nd Creates a snapshot whose contents are as specified in the stream provided on standard input.
|
||
.Sh SYNOPSIS
|
||
.Nm
|
||
.Cm receive
|
||
.Op Fl FhMnsuv
|
||
.Op Fl o Sy origin Ns = Ns Ar snapshot
|
||
.Op Fl o Ar property Ns = Ns Ar value
|
||
.Op Fl x Ar property
|
||
.Ar filesystem Ns | Ns Ar volume Ns | Ns Ar snapshot
|
||
.Nm
|
||
.Cm receive
|
||
.Op Fl FhMnsuv
|
||
.Op Fl d Ns | Ns Fl e
|
||
.Op Fl o Sy origin Ns = Ns Ar snapshot
|
||
.Op Fl o Ar property Ns = Ns Ar value
|
||
.Op Fl x Ar property
|
||
.Ar filesystem
|
||
.Nm
|
||
.Cm receive
|
||
.Fl A
|
||
.Ar filesystem Ns | Ns Ar volume
|
||
.Sh DESCRIPTION
|
||
.Bl -tag -width ""
|
||
.It Xo
|
||
.Nm
|
||
.Cm receive
|
||
.Op Fl FhMnsuv
|
||
.Op Fl o Sy origin Ns = Ns Ar snapshot
|
||
.Op Fl o Ar property Ns = Ns Ar value
|
||
.Op Fl x Ar property
|
||
.Ar filesystem Ns | Ns Ar volume Ns | Ns Ar snapshot
|
||
.Xc
|
||
.It Xo
|
||
.Nm
|
||
.Cm receive
|
||
.Op Fl FhMnsuv
|
||
.Op Fl d Ns | Ns Fl e
|
||
.Op Fl o Sy origin Ns = Ns Ar snapshot
|
||
.Op Fl o Ar property Ns = Ns Ar value
|
||
.Op Fl x Ar property
|
||
.Ar filesystem
|
||
.Xc
|
||
Creates a snapshot whose contents are as specified in the stream provided on
|
||
standard input.
|
||
If a full stream is received, then a new file system is created as well.
|
||
Streams are created using the
|
||
.Nm zfs Cm send
|
||
subcommand, which by default creates a full stream.
|
||
.Nm zfs Cm recv
|
||
can be used as an alias for
|
||
.Nm zfs Cm receive.
|
||
.Pp
|
||
If an incremental stream is received, then the destination file system must
|
||
already exist, and its most recent snapshot must match the incremental stream's
|
||
source.
|
||
For
|
||
.Sy zvols ,
|
||
the destination device link is destroyed and recreated, which means the
|
||
.Sy zvol
|
||
cannot be accessed during the
|
||
.Cm receive
|
||
operation.
|
||
.Pp
|
||
When a snapshot replication package stream that is generated by using the
|
||
.Nm zfs Cm send Fl R
|
||
command is received, any snapshots that do not exist on the sending location are
|
||
destroyed by using the
|
||
.Nm zfs Cm destroy Fl d
|
||
command.
|
||
.Pp
|
||
The ability to send and receive deduplicated send streams has been removed.
|
||
However, a deduplicated send stream created with older software can be converted
|
||
to a regular (non-deduplicated) stream by using the
|
||
.Nm zstream Cm redup
|
||
command.
|
||
.Pp
|
||
If
|
||
.Fl o Em property Ns = Ns Ar value
|
||
or
|
||
.Fl x Em property
|
||
is specified, it applies to the effective value of the property throughout
|
||
the entire subtree of replicated datasets. Effective property values will be
|
||
set (
|
||
.Fl o
|
||
) or inherited (
|
||
.Fl x
|
||
) on the topmost in the replicated subtree. In descendant datasets, if the
|
||
property is set by the send stream, it will be overridden by forcing the
|
||
property to be inherited from the top‐most file system. Received properties
|
||
are retained in spite of being overridden and may be restored with
|
||
.Nm zfs Cm inherit Fl S .
|
||
Specifying
|
||
.Fl o Sy origin Ns = Ns Em snapshot
|
||
is a special case because, even if
|
||
.Sy origin
|
||
is a read-only property and cannot be set, it's allowed to receive the send
|
||
stream as a clone of the given snapshot.
|
||
.Pp
|
||
Raw encrypted send streams (created with
|
||
.Nm zfs Cm send Fl w
|
||
) may only be received as is, and cannot be re-encrypted, decrypted, or
|
||
recompressed by the receive process. Unencrypted streams can be received as
|
||
encrypted datasets, either through inheritance or by specifying encryption
|
||
parameters with the
|
||
.Fl o
|
||
options. Note that the
|
||
.Sy keylocation
|
||
property cannot be overridden to
|
||
.Sy prompt
|
||
during a receive. This is because the receive process itself is already using
|
||
stdin for the send stream. Instead, the property can be overridden after the
|
||
receive completes.
|
||
.Pp
|
||
The added security provided by raw sends adds some restrictions to the send
|
||
and receive process. ZFS will not allow a mix of raw receives and non-raw
|
||
receives. Specifically, any raw incremental receives that are attempted after
|
||
a non-raw receive will fail. Non-raw receives do not have this restriction and,
|
||
therefore, are always possible. Because of this, it is best practice to always
|
||
use either raw sends for their security benefits or non-raw sends for their
|
||
flexibility when working with encrypted datasets, but not a combination.
|
||
.Pp
|
||
The reason for this restriction stems from the inherent restrictions of the
|
||
AEAD ciphers that ZFS uses to encrypt data. When using ZFS native encryption,
|
||
each block of data is encrypted against a randomly generated number known as
|
||
the "initialization vector" (IV), which is stored in the filesystem metadata.
|
||
This number is required by the encryption algorithms whenever the data is to
|
||
be decrypted. Together, all of the IVs provided for all of the blocks in a
|
||
given snapshot are collectively called an "IV set". When ZFS performs a raw
|
||
send, the IV set is transferred from the source to the destination in the send
|
||
stream. When ZFS performs a non-raw send, the data is decrypted by the source
|
||
system and re-encrypted by the destination system, creating a snapshot with
|
||
effectively the same data, but a different IV set. In order for decryption to
|
||
work after a raw send, ZFS must ensure that the IV set used on both the source
|
||
and destination side match. When an incremental raw receive is performed on
|
||
top of an existing snapshot, ZFS will check to confirm that the "from"
|
||
snapshot on both the source and destination were using the same IV set,
|
||
ensuring the new IV set is consistent.
|
||
.Pp
|
||
The name of the snapshot
|
||
.Pq and file system, if a full stream is received
|
||
that this subcommand creates depends on the argument type and the use of the
|
||
.Fl d
|
||
or
|
||
.Fl e
|
||
options.
|
||
.Pp
|
||
If the argument is a snapshot name, the specified
|
||
.Ar snapshot
|
||
is created.
|
||
If the argument is a file system or volume name, a snapshot with the same name
|
||
as the sent snapshot is created within the specified
|
||
.Ar filesystem
|
||
or
|
||
.Ar volume .
|
||
If neither of the
|
||
.Fl d
|
||
or
|
||
.Fl e
|
||
options are specified, the provided target snapshot name is used exactly as
|
||
provided.
|
||
.Pp
|
||
The
|
||
.Fl d
|
||
and
|
||
.Fl e
|
||
options cause the file system name of the target snapshot to be determined by
|
||
appending a portion of the sent snapshot's name to the specified target
|
||
.Ar filesystem .
|
||
If the
|
||
.Fl d
|
||
option is specified, all but the first element of the sent snapshot's file
|
||
system path
|
||
.Pq usually the pool name
|
||
is used and any required intermediate file systems within the specified one are
|
||
created.
|
||
If the
|
||
.Fl e
|
||
option is specified, then only the last element of the sent snapshot's file
|
||
system name
|
||
.Pq i.e. the name of the source file system itself
|
||
is used as the target file system name.
|
||
.Bl -tag -width "-F"
|
||
.It Fl F
|
||
Force a rollback of the file system to the most recent snapshot before
|
||
performing the receive operation.
|
||
If receiving an incremental replication stream
|
||
.Po for example, one generated by
|
||
.Nm zfs Cm send Fl R Op Fl i Ns | Ns Fl I
|
||
.Pc ,
|
||
destroy snapshots and file systems that do not exist on the sending side.
|
||
.It Fl d
|
||
Discard the first element of the sent snapshot's file system name, using the
|
||
remaining elements to determine the name of the target file system for the new
|
||
snapshot as described in the paragraph above.
|
||
.It Fl e
|
||
Discard all but the last element of the sent snapshot's file system name, using
|
||
that element to determine the name of the target file system for the new
|
||
snapshot as described in the paragraph above.
|
||
.It Fl h
|
||
Skip the receive of holds. There is no effect if holds are not sent.
|
||
.It Fl M
|
||
Force an unmount of the file system while receiving a snapshot.
|
||
This option is not supported on Linux.
|
||
.It Fl n
|
||
Do not actually receive the stream.
|
||
This can be useful in conjunction with the
|
||
.Fl v
|
||
option to verify the name the receive operation would use.
|
||
.It Fl o Sy origin Ns = Ns Ar snapshot
|
||
Forces the stream to be received as a clone of the given snapshot.
|
||
If the stream is a full send stream, this will create the filesystem
|
||
described by the stream as a clone of the specified snapshot.
|
||
Which snapshot was specified will not affect the success or failure of the
|
||
receive, as long as the snapshot does exist.
|
||
If the stream is an incremental send stream, all the normal verification will be
|
||
performed.
|
||
.It Fl o Em property Ns = Ns Ar value
|
||
Sets the specified property as if the command
|
||
.Nm zfs Cm set Em property Ns = Ns Ar value
|
||
was invoked immediately before the receive. When receiving a stream from
|
||
.Nm zfs Cm send Fl R ,
|
||
causes the property to be inherited by all descendant datasets, as through
|
||
.Nm zfs Cm inherit Em property
|
||
was run on any descendant datasets that have this property set on the
|
||
sending system.
|
||
.Pp
|
||
Any editable property can be set at receive time. Set-once properties bound
|
||
to the received data, such as
|
||
.Sy normalization
|
||
and
|
||
.Sy casesensitivity ,
|
||
cannot be set at receive time even when the datasets are newly created by
|
||
.Nm zfs Cm receive .
|
||
Additionally both settable properties
|
||
.Sy version
|
||
and
|
||
.Sy volsize
|
||
cannot be set at receive time.
|
||
.Pp
|
||
The
|
||
.Fl o
|
||
option may be specified multiple times, for different properties. An error
|
||
results if the same property is specified in multiple
|
||
.Fl o
|
||
or
|
||
.Fl x
|
||
options.
|
||
.Pp
|
||
The
|
||
.Fl o
|
||
option may also be used to override encryption properties upon initial
|
||
receive. This allows unencrypted streams to be received as encrypted datasets.
|
||
To cause the received dataset (or root dataset of a recursive stream) to be
|
||
received as an encryption root, specify encryption properties in the same
|
||
manner as is required for
|
||
.Nm
|
||
.Cm create .
|
||
For instance:
|
||
.Bd -literal
|
||
# zfs send tank/test@snap1 | zfs recv -o encryption=on -o keyformat=passphrase -o keylocation=file:///path/to/keyfile
|
||
.Ed
|
||
.Pp
|
||
Note that
|
||
.Op Fl o Ar keylocation Ns = Ns Ar prompt
|
||
may not be specified here, since stdin is already being utilized for the send
|
||
stream. Once the receive has completed, you can use
|
||
.Nm
|
||
.Cm set
|
||
to change this setting after the fact. Similarly, you can receive a dataset as
|
||
an encrypted child by specifying
|
||
.Op Fl x Ar encryption
|
||
to force the property to be inherited. Overriding encryption properties (except
|
||
for
|
||
.Sy keylocation Ns )
|
||
is not possible with raw send streams.
|
||
.It Fl s
|
||
If the receive is interrupted, save the partially received state, rather
|
||
than deleting it.
|
||
Interruption may be due to premature termination of the stream
|
||
.Po e.g. due to network failure or failure of the remote system
|
||
if the stream is being read over a network connection
|
||
.Pc ,
|
||
a checksum error in the stream, termination of the
|
||
.Nm zfs Cm receive
|
||
process, or unclean shutdown of the system.
|
||
.Pp
|
||
The receive can be resumed with a stream generated by
|
||
.Nm zfs Cm send Fl t Ar token ,
|
||
where the
|
||
.Ar token
|
||
is the value of the
|
||
.Sy receive_resume_token
|
||
property of the filesystem or volume which is received into.
|
||
.Pp
|
||
To use this flag, the storage pool must have the
|
||
.Sy extensible_dataset
|
||
feature enabled.
|
||
See
|
||
.Xr zpool-features 5
|
||
for details on ZFS feature flags.
|
||
.It Fl u
|
||
File system that is associated with the received stream is not mounted.
|
||
.It Fl v
|
||
Print verbose information about the stream and the time required to perform the
|
||
receive operation.
|
||
.It Fl x Em property
|
||
Ensures that the effective value of the specified property after the
|
||
receive is unaffected by the value of that property in the send stream (if any),
|
||
as if the property had been excluded from the send stream.
|
||
.Pp
|
||
If the specified property is not present in the send stream, this option does
|
||
nothing.
|
||
.Pp
|
||
If a received property needs to be overridden, the effective value will be
|
||
set or inherited, depending on whether the property is inheritable or not.
|
||
.Pp
|
||
In the case of an incremental update,
|
||
.Fl x
|
||
leaves any existing local setting or explicit inheritance unchanged.
|
||
.Pp
|
||
All
|
||
.Fl o
|
||
restrictions (e.g. set-once) apply equally to
|
||
.Fl x .
|
||
.El
|
||
.It Xo
|
||
.Nm
|
||
.Cm receive
|
||
.Fl A
|
||
.Ar filesystem Ns | Ns Ar volume
|
||
.Xc
|
||
Abort an interrupted
|
||
.Nm zfs Cm receive Fl s ,
|
||
deleting its saved partially received state.
|
||
.El
|
||
.Sh SEE ALSO
|
||
.Xr zfs-send 8
|
||
.Xr zstream 8
|