2020-03-10 17:12:50 +03:00
|
|
|
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
|
|
|
|
From: Dietmar Maurer <dietmar@proxmox.com>
|
2020-04-07 17:53:19 +03:00
|
|
|
Date: Mon, 6 Apr 2020 12:17:01 +0200
|
|
|
|
Subject: [PATCH] PVE-Backup: pbs-restore - new command to restore from proxmox
|
|
|
|
backup server
|
2020-03-10 17:12:50 +03:00
|
|
|
|
2020-05-30 17:06:01 +03:00
|
|
|
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
|
2020-03-10 17:12:50 +03:00
|
|
|
---
|
2021-02-11 19:11:11 +03:00
|
|
|
meson.build | 4 +
|
update submodule and patches to 7.1.0
Notable changes:
* The only big change is the switch to using a custom QIOChannel for
savevm-async, because the previously used QEMUFileOps was dropped.
Changes to the current implementation:
* Switch to vector based methods as required for an IO channel. For
short reads the passed-in IO vector is stuffed with zeroes at the
end, just to be sure.
* For reading: The documentation in include/io/channel.h states that
at least one byte should be read, so also error out when whe are
at the very end instead of returning 0.
* For reading: Fix off-by-one error when request goes beyond end.
The wrong code piece was:
if ((pos + size) > maxlen) {
size = maxlen - pos - 1;
}
Previously, the last byte would not be read. It's actually
possible to get a snapshot .raw file that has content all the way
up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
trailing zero bytes (I wrote a script to do it).
Luckily, it didn't cause a real issue, because qemu_loadvm_state()
is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
section. The buffer for reading it is simply freed up afterwards
and the function will assume that it read the whole section, even
if that's not the case.
* For writing: Make use of the generated blk_pwritev() wrapper
instead of manually wrapping the coroutine to simplify and save a
few lines.
* Adapt to changed interfaces for blk_{pread,pwrite}:
* a9262f551e ("block: Change blk_{pread,pwrite}() param order")
* 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
* bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
Those changes especially affected the qemu-img dd patches, because
the context also changed, but also some of our block drivers used
the functions.
* Drop qemu-common.h include: it got renamed after essentially
everything was moved to other headers. The only remaining user I
could find for things dropped from the header between 7.0 and 7.1
was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
already includes the header to which the function was moved.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
|
|
|
pbs-restore.c | 223 ++++++++++++++++++++++++++++++++++++++++++++++++++
|
|
|
|
2 files changed, 227 insertions(+)
|
2020-03-10 17:12:50 +03:00
|
|
|
create mode 100644 pbs-restore.c
|
|
|
|
|
2021-02-11 19:11:11 +03:00
|
|
|
diff --git a/meson.build b/meson.build
|
2023-05-15 16:39:54 +03:00
|
|
|
index d307d8eabf..afd105001e 100644
|
2021-02-11 19:11:11 +03:00
|
|
|
--- a/meson.build
|
|
|
|
+++ b/meson.build
|
2023-05-15 16:39:54 +03:00
|
|
|
@@ -3652,6 +3652,10 @@ if have_tools
|
2021-11-18 10:05:38 +03:00
|
|
|
vma = executable('vma', files('vma.c', 'vma-reader.c') + genh,
|
2021-02-11 19:11:11 +03:00
|
|
|
dependencies: [authz, block, crypto, io, qom], install: true)
|
2020-03-10 17:12:50 +03:00
|
|
|
|
2021-11-18 10:05:38 +03:00
|
|
|
+ pbs_restore = executable('pbs-restore', files('pbs-restore.c') + genh,
|
2021-02-11 19:11:11 +03:00
|
|
|
+ dependencies: [authz, block, crypto, io, qom,
|
|
|
|
+ libproxmox_backup_qemu], install: true)
|
|
|
|
+
|
|
|
|
subdir('storage-daemon')
|
|
|
|
subdir('contrib/rdmacm-mux')
|
|
|
|
subdir('contrib/elf2dmp')
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/pbs-restore.c b/pbs-restore.c
|
|
|
|
new file mode 100644
|
update submodule and patches to 7.1.0
Notable changes:
* The only big change is the switch to using a custom QIOChannel for
savevm-async, because the previously used QEMUFileOps was dropped.
Changes to the current implementation:
* Switch to vector based methods as required for an IO channel. For
short reads the passed-in IO vector is stuffed with zeroes at the
end, just to be sure.
* For reading: The documentation in include/io/channel.h states that
at least one byte should be read, so also error out when whe are
at the very end instead of returning 0.
* For reading: Fix off-by-one error when request goes beyond end.
The wrong code piece was:
if ((pos + size) > maxlen) {
size = maxlen - pos - 1;
}
Previously, the last byte would not be read. It's actually
possible to get a snapshot .raw file that has content all the way
up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
trailing zero bytes (I wrote a script to do it).
Luckily, it didn't cause a real issue, because qemu_loadvm_state()
is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
section. The buffer for reading it is simply freed up afterwards
and the function will assume that it read the whole section, even
if that's not the case.
* For writing: Make use of the generated blk_pwritev() wrapper
instead of manually wrapping the coroutine to simplify and save a
few lines.
* Adapt to changed interfaces for blk_{pread,pwrite}:
* a9262f551e ("block: Change blk_{pread,pwrite}() param order")
* 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
* bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
Those changes especially affected the qemu-img dd patches, because
the context also changed, but also some of our block drivers used
the functions.
* Drop qemu-common.h include: it got renamed after essentially
everything was moved to other headers. The only remaining user I
could find for things dropped from the header between 7.0 and 7.1
was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
already includes the header to which the function was moved.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
|
|
|
index 0000000000..2f834cf42e
|
2020-03-10 17:12:50 +03:00
|
|
|
--- /dev/null
|
|
|
|
+++ b/pbs-restore.c
|
update submodule and patches to 7.1.0
Notable changes:
* The only big change is the switch to using a custom QIOChannel for
savevm-async, because the previously used QEMUFileOps was dropped.
Changes to the current implementation:
* Switch to vector based methods as required for an IO channel. For
short reads the passed-in IO vector is stuffed with zeroes at the
end, just to be sure.
* For reading: The documentation in include/io/channel.h states that
at least one byte should be read, so also error out when whe are
at the very end instead of returning 0.
* For reading: Fix off-by-one error when request goes beyond end.
The wrong code piece was:
if ((pos + size) > maxlen) {
size = maxlen - pos - 1;
}
Previously, the last byte would not be read. It's actually
possible to get a snapshot .raw file that has content all the way
up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
trailing zero bytes (I wrote a script to do it).
Luckily, it didn't cause a real issue, because qemu_loadvm_state()
is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
section. The buffer for reading it is simply freed up afterwards
and the function will assume that it read the whole section, even
if that's not the case.
* For writing: Make use of the generated blk_pwritev() wrapper
instead of manually wrapping the coroutine to simplify and save a
few lines.
* Adapt to changed interfaces for blk_{pread,pwrite}:
* a9262f551e ("block: Change blk_{pread,pwrite}() param order")
* 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
* bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
Those changes especially affected the qemu-img dd patches, because
the context also changed, but also some of our block drivers used
the functions.
* Drop qemu-common.h include: it got renamed after essentially
everything was moved to other headers. The only remaining user I
could find for things dropped from the header between 7.0 and 7.1
was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
already includes the header to which the function was moved.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
|
|
|
@@ -0,0 +1,223 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
+/*
|
|
|
|
+ * Qemu image restore helper for Proxmox Backup
|
|
|
|
+ *
|
|
|
|
+ * Copyright (C) 2019 Proxmox Server Solutions
|
|
|
|
+ *
|
|
|
|
+ * Authors:
|
|
|
|
+ * Dietmar Maurer (dietmar@proxmox.com)
|
|
|
|
+ *
|
|
|
|
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
|
|
|
|
+ * See the COPYING file in the top-level directory.
|
|
|
|
+ *
|
|
|
|
+ */
|
|
|
|
+
|
|
|
|
+#include "qemu/osdep.h"
|
|
|
|
+#include <glib.h>
|
|
|
|
+#include <getopt.h>
|
|
|
|
+#include <string.h>
|
|
|
|
+
|
|
|
|
+#include "qemu/module.h"
|
|
|
|
+#include "qemu/error-report.h"
|
|
|
|
+#include "qemu/main-loop.h"
|
|
|
|
+#include "qemu/cutils.h"
|
|
|
|
+#include "qapi/error.h"
|
|
|
|
+#include "qapi/qmp/qdict.h"
|
|
|
|
+#include "sysemu/block-backend.h"
|
|
|
|
+
|
|
|
|
+#include <proxmox-backup-qemu.h>
|
|
|
|
+
|
|
|
|
+static void help(void)
|
|
|
|
+{
|
|
|
|
+ const char *help_msg =
|
|
|
|
+ "usage: pbs-restore [--repository <repo>] snapshot archive-name target [command options]\n"
|
|
|
|
+ ;
|
|
|
|
+
|
|
|
|
+ printf("%s", help_msg);
|
|
|
|
+ exit(1);
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+typedef struct CallbackData {
|
|
|
|
+ BlockBackend *target;
|
|
|
|
+ uint64_t last_offset;
|
|
|
|
+ bool skip_zero;
|
|
|
|
+} CallbackData;
|
|
|
|
+
|
|
|
|
+static int write_callback(
|
|
|
|
+ void *callback_data_ptr,
|
|
|
|
+ uint64_t offset,
|
|
|
|
+ const unsigned char *data,
|
|
|
|
+ uint64_t data_len)
|
|
|
|
+{
|
|
|
|
+ int res = -1;
|
|
|
|
+
|
|
|
|
+ CallbackData *callback_data = (CallbackData *)callback_data_ptr;
|
|
|
|
+
|
|
|
|
+ uint64_t last_offset = callback_data->last_offset;
|
|
|
|
+ if (offset > last_offset) callback_data->last_offset = offset;
|
|
|
|
+
|
|
|
|
+ if (data == NULL) {
|
|
|
|
+ if (callback_data->skip_zero && offset > last_offset) {
|
|
|
|
+ return 0;
|
|
|
|
+ }
|
|
|
|
+ res = blk_pwrite_zeroes(callback_data->target, offset, data_len, 0);
|
|
|
|
+ } else {
|
update submodule and patches to 7.1.0
Notable changes:
* The only big change is the switch to using a custom QIOChannel for
savevm-async, because the previously used QEMUFileOps was dropped.
Changes to the current implementation:
* Switch to vector based methods as required for an IO channel. For
short reads the passed-in IO vector is stuffed with zeroes at the
end, just to be sure.
* For reading: The documentation in include/io/channel.h states that
at least one byte should be read, so also error out when whe are
at the very end instead of returning 0.
* For reading: Fix off-by-one error when request goes beyond end.
The wrong code piece was:
if ((pos + size) > maxlen) {
size = maxlen - pos - 1;
}
Previously, the last byte would not be read. It's actually
possible to get a snapshot .raw file that has content all the way
up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
trailing zero bytes (I wrote a script to do it).
Luckily, it didn't cause a real issue, because qemu_loadvm_state()
is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
section. The buffer for reading it is simply freed up afterwards
and the function will assume that it read the whole section, even
if that's not the case.
* For writing: Make use of the generated blk_pwritev() wrapper
instead of manually wrapping the coroutine to simplify and save a
few lines.
* Adapt to changed interfaces for blk_{pread,pwrite}:
* a9262f551e ("block: Change blk_{pread,pwrite}() param order")
* 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
* bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
Those changes especially affected the qemu-img dd patches, because
the context also changed, but also some of our block drivers used
the functions.
* Drop qemu-common.h include: it got renamed after essentially
everything was moved to other headers. The only remaining user I
could find for things dropped from the header between 7.0 and 7.1
was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
already includes the header to which the function was moved.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
|
|
|
+ res = blk_pwrite(callback_data->target, offset, data_len, data, 0);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (res < 0) {
|
|
|
|
+ fprintf(stderr, "blk_pwrite failed at offset %ld length %ld (%d) - %s\n", offset, data_len, res, strerror(-res));
|
|
|
|
+ return res;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ return 0;
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+int main(int argc, char **argv)
|
|
|
|
+{
|
|
|
|
+ Error *main_loop_err = NULL;
|
|
|
|
+ const char *format = "raw";
|
|
|
|
+ const char *repository = NULL;
|
|
|
|
+ const char *keyfile = NULL;
|
|
|
|
+ int verbose = false;
|
|
|
|
+ bool skip_zero = false;
|
|
|
|
+
|
|
|
|
+ error_init(argv[0]);
|
|
|
|
+
|
2020-05-30 16:24:11 +03:00
|
|
|
+ for (;;) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ static const struct option long_options[] = {
|
|
|
|
+ {"help", no_argument, 0, 'h'},
|
|
|
|
+ {"skip-zero", no_argument, 0, 'S'},
|
|
|
|
+ {"verbose", no_argument, 0, 'v'},
|
|
|
|
+ {"format", required_argument, 0, 'f'},
|
|
|
|
+ {"repository", required_argument, 0, 'r'},
|
|
|
|
+ {"keyfile", required_argument, 0, 'k'},
|
|
|
|
+ {0, 0, 0, 0}
|
|
|
|
+ };
|
|
|
|
+ int c = getopt_long(argc, argv, "hvf:r:k:", long_options, NULL);
|
|
|
|
+ if (c == -1) {
|
|
|
|
+ break;
|
|
|
|
+ }
|
2020-05-30 16:24:11 +03:00
|
|
|
+ switch (c) {
|
|
|
|
+ case ':':
|
|
|
|
+ fprintf(stderr, "missing argument for option '%s'\n", argv[optind - 1]);
|
|
|
|
+ return -1;
|
|
|
|
+ case '?':
|
|
|
|
+ fprintf(stderr, "unrecognized option '%s'\n", argv[optind - 1]);
|
|
|
|
+ return -1;
|
|
|
|
+ case 'f':
|
|
|
|
+ format = g_strdup(argv[optind - 1]);
|
|
|
|
+ break;
|
|
|
|
+ case 'r':
|
|
|
|
+ repository = g_strdup(argv[optind - 1]);
|
|
|
|
+ break;
|
|
|
|
+ case 'k':
|
|
|
|
+ keyfile = g_strdup(argv[optind - 1]);
|
|
|
|
+ break;
|
|
|
|
+ case 'v':
|
|
|
|
+ verbose = true;
|
|
|
|
+ break;
|
|
|
|
+ case 'S':
|
|
|
|
+ skip_zero = true;
|
|
|
|
+ break;
|
|
|
|
+ case 'h':
|
|
|
|
+ help();
|
|
|
|
+ return 0;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (optind >= argc - 2) {
|
|
|
|
+ fprintf(stderr, "missing arguments\n");
|
|
|
|
+ help();
|
|
|
|
+ return -1;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (repository == NULL) {
|
|
|
|
+ repository = getenv("PBS_REPOSITORY");
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (repository == NULL) {
|
|
|
|
+ fprintf(stderr, "no repository specified\n");
|
|
|
|
+ help();
|
|
|
|
+ return -1;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ char *snapshot = argv[optind++];
|
|
|
|
+ char *archive_name = argv[optind++];
|
|
|
|
+ char *target = argv[optind++];
|
|
|
|
+
|
|
|
|
+ const char *password = getenv("PBS_PASSWORD");
|
|
|
|
+ const char *fingerprint = getenv("PBS_FINGERPRINT");
|
|
|
|
+ const char *key_password = getenv("PBS_ENCRYPTION_PASSWORD");
|
|
|
|
+
|
|
|
|
+ if (qemu_init_main_loop(&main_loop_err)) {
|
|
|
|
+ g_error("%s", error_get_pretty(main_loop_err));
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ bdrv_init();
|
|
|
|
+ module_call_init(MODULE_INIT_QOM);
|
|
|
|
+
|
2020-05-30 17:06:01 +03:00
|
|
|
+ if (verbose) {
|
|
|
|
+ fprintf(stderr, "connecting to repository '%s'\n", repository);
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_error = NULL;
|
2021-03-03 12:56:02 +03:00
|
|
|
+ ProxmoxRestoreHandle *conn = proxmox_restore_new(
|
2020-03-10 17:12:50 +03:00
|
|
|
+ repository, snapshot, password, keyfile, key_password, fingerprint, &pbs_error);
|
|
|
|
+ if (conn == NULL) {
|
|
|
|
+ fprintf(stderr, "restore failed: %s\n", pbs_error);
|
|
|
|
+ return -1;
|
|
|
|
+ }
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ int res = proxmox_restore_connect(conn, &pbs_error);
|
|
|
|
+ if (res < 0 || pbs_error) {
|
|
|
|
+ fprintf(stderr, "restore failed (connection error): %s\n", pbs_error);
|
|
|
|
+ return -1;
|
|
|
|
+ }
|
|
|
|
+
|
2020-03-10 17:12:50 +03:00
|
|
|
+ QDict *options = qdict_new();
|
|
|
|
+
|
|
|
|
+ if (format) {
|
|
|
|
+ qdict_put_str(options, "driver", format);
|
|
|
|
+ }
|
|
|
|
+
|
2020-05-30 17:06:01 +03:00
|
|
|
+
|
|
|
|
+ if (verbose) {
|
|
|
|
+ fprintf(stderr, "open block backend for target '%s'\n", target);
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+ Error *local_err = NULL;
|
|
|
|
+ int flags = BDRV_O_RDWR;
|
|
|
|
+ BlockBackend *blk = blk_new_open(target, NULL, options, flags, &local_err);
|
|
|
|
+ if (!blk) {
|
|
|
|
+ fprintf(stderr, "%s\n", error_get_pretty(local_err));
|
|
|
|
+ return -1;
|
|
|
|
+ }
|
|
|
|
+
|
2020-05-30 16:24:11 +03:00
|
|
|
+ CallbackData *callback_data = calloc(sizeof(CallbackData), 1);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ callback_data->target = blk;
|
|
|
|
+ callback_data->skip_zero = skip_zero;
|
|
|
|
+ callback_data->last_offset = 0;
|
|
|
|
+
|
|
|
|
+ // blk_set_enable_write_cache(blk, !writethrough);
|
|
|
|
+
|
2020-05-30 17:06:01 +03:00
|
|
|
+ if (verbose) {
|
|
|
|
+ fprintf(stderr, "starting to restore snapshot '%s'\n", snapshot);
|
2020-06-02 14:20:56 +03:00
|
|
|
+ fflush(stderr); // ensure we do not get printed after the progress log
|
2020-05-30 17:06:01 +03:00
|
|
|
+ }
|
2021-03-03 12:56:02 +03:00
|
|
|
+ res = proxmox_restore_image(
|
2020-03-10 17:12:50 +03:00
|
|
|
+ conn,
|
|
|
|
+ archive_name,
|
|
|
|
+ write_callback,
|
|
|
|
+ callback_data,
|
|
|
|
+ &pbs_error,
|
|
|
|
+ verbose);
|
|
|
|
+
|
|
|
|
+ proxmox_restore_disconnect(conn);
|
2021-02-24 16:34:47 +03:00
|
|
|
+ blk_unref(blk);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ if (res < 0) {
|
|
|
|
+ fprintf(stderr, "restore failed: %s\n", pbs_error);
|
|
|
|
+ return -1;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ return 0;
|
|
|
|
+}
|