Add ability to recompress send streams with new compression algorithm

As new compression algorithms are added to ZFS, it could be useful for 
people to recompress data with new algorithms. There is currently no 
mechanism to do this aside from copying the data manually into a new 
filesystem with the new algorithm enabled. This tool allows the 
transformation to happen through zfs send, allowing it to be done 
efficiently to remote systems and in an incremental fashion.

A new zstream command is added that decompresses WRITE records and 
then recompresses them with a provided algorithm, and then re-emits 
the modified send stream. It may also be possible to re-compress 
embedded block pointers, but that was not attempted for the initial 
version.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes #14106
This commit is contained in:
Paul Dagnelie
2022-11-10 15:23:46 -08:00
committed by GitHub
parent e9ab9e512c
commit 9f4ede63d2
7 changed files with 451 additions and 6 deletions
+25
View File
@@ -43,6 +43,10 @@
.Nm
.Cm token
.Ar resume_token
.Nm
.Cm recompress
.Op Fl l Ar level
.Ar algorithm
.
.Sh DESCRIPTION
The
@@ -149,6 +153,27 @@ Therefore, a deduplicated send stream can be received by running:
Verbose.
Print summary of converted records.
.El
.It Xo
.Nm
.Cm recompress
.Op Fl l Ar level
.Ar algorithm
.Xc
Recompresses a send stream, provided on standard input, using the provided
algorithm and optional level, and writes the modified stream to standard output.
All WRITE records in the send stream will be recompressed, unless they fail
to result in size reduction compared to being left uncompressed.
The provided algorithm can be any valid value to the
.Nm compress
property.
Note that encrypted send streams cannot be recompressed.
.Bl -tag -width "-l"
.It Fl l Ar level
Specifies compression level.
Only needed for algorithms where the level is not implied as part of the name
of the algorithm (e.g. gzip-3 does not require it, while zstd does, if a
non-default level is desired).
.El
.El
.
.Sh EXAMPLES