2020-03-10 17:12:50 +03:00
|
|
|
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
|
|
|
|
From: Dietmar Maurer <dietmar@proxmox.com>
|
2020-04-07 17:53:19 +03:00
|
|
|
Date: Mon, 6 Apr 2020 12:16:59 +0200
|
|
|
|
Subject: [PATCH] PVE-Backup: proxmox backup patches for qemu
|
2020-03-10 17:12:50 +03:00
|
|
|
|
2021-03-03 12:56:02 +03:00
|
|
|
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
|
|
|
|
[PVE-Backup: avoid coroutines to fix AIO freeze, cleanups]
|
|
|
|
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
|
2022-01-13 12:34:33 +03:00
|
|
|
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
|
2022-12-14 17:16:32 +03:00
|
|
|
[FE: add new force parameter to job_cancel_sync calls
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
adapt for new job lock mechanism replacing AioContext locks
|
|
|
|
adapt to QAPI changes]
|
2022-12-14 17:16:32 +03:00
|
|
|
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
|
2020-03-10 17:12:50 +03:00
|
|
|
---
|
2021-02-11 19:11:11 +03:00
|
|
|
block/meson.build | 5 +
|
2020-04-07 17:53:19 +03:00
|
|
|
block/monitor/block-hmp-cmds.c | 33 ++
|
|
|
|
blockdev.c | 1 +
|
2021-10-11 14:55:34 +03:00
|
|
|
hmp-commands-info.hx | 14 +
|
2020-04-07 17:53:19 +03:00
|
|
|
hmp-commands.hx | 29 +
|
|
|
|
include/monitor/hmp.h | 3 +
|
2021-02-11 19:11:11 +03:00
|
|
|
meson.build | 1 +
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
monitor/hmp-cmds.c | 45 ++
|
|
|
|
proxmox-backup-client.c | 176 +++++++
|
|
|
|
proxmox-backup-client.h | 59 +++
|
|
|
|
pve-backup.c | 938 +++++++++++++++++++++++++++++++++
|
2020-04-07 17:53:19 +03:00
|
|
|
qapi/block-core.json | 109 ++++
|
|
|
|
qapi/common.json | 13 +
|
2021-02-11 19:11:11 +03:00
|
|
|
qapi/machine.json | 15 +-
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
14 files changed, 1428 insertions(+), 13 deletions(-)
|
2020-03-10 17:12:50 +03:00
|
|
|
create mode 100644 proxmox-backup-client.c
|
|
|
|
create mode 100644 proxmox-backup-client.h
|
|
|
|
create mode 100644 pve-backup.c
|
|
|
|
|
2021-02-11 19:11:11 +03:00
|
|
|
diff --git a/block/meson.build b/block/meson.build
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
index f580f95395..5bcebb934b 100644
|
2021-02-11 19:11:11 +03:00
|
|
|
--- a/block/meson.build
|
|
|
|
+++ b/block/meson.build
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -49,6 +49,11 @@ block_ss.add(files(
|
2021-05-27 13:43:32 +03:00
|
|
|
), zstd, zlib, gnutls)
|
2020-04-07 17:53:19 +03:00
|
|
|
|
2021-02-11 19:11:11 +03:00
|
|
|
block_ss.add(files('../vma-writer.c'), libuuid)
|
|
|
|
+block_ss.add(files(
|
|
|
|
+ '../proxmox-backup-client.c',
|
|
|
|
+ '../pve-backup.c',
|
|
|
|
+), libproxmox_backup_qemu)
|
|
|
|
+
|
2020-04-07 17:53:19 +03:00
|
|
|
|
2021-02-11 19:11:11 +03:00
|
|
|
softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
|
2022-12-14 17:16:32 +03:00
|
|
|
softmmu_ss.add(files('block-ram-registrar.c'))
|
2020-04-07 17:53:19 +03:00
|
|
|
diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
|
2023-05-15 16:39:54 +03:00
|
|
|
index ca2599de44..d50e99df26 100644
|
2020-04-07 17:53:19 +03:00
|
|
|
--- a/block/monitor/block-hmp-cmds.c
|
|
|
|
+++ b/block/monitor/block-hmp-cmds.c
|
2023-05-15 16:39:54 +03:00
|
|
|
@@ -1029,3 +1029,36 @@ void hmp_change_medium(Monitor *mon, const char *device, const char *target,
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
qmp_blockdev_change_medium(device, NULL, target, arg, true, force,
|
|
|
|
!!read_only, read_only_mode, errp);
|
2020-04-07 17:53:19 +03:00
|
|
|
}
|
|
|
|
+
|
|
|
|
+void hmp_backup_cancel(Monitor *mon, const QDict *qdict)
|
|
|
|
+{
|
|
|
|
+ Error *error = NULL;
|
|
|
|
+
|
|
|
|
+ qmp_backup_cancel(&error);
|
|
|
|
+
|
|
|
|
+ hmp_handle_error(mon, error);
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+void hmp_backup(Monitor *mon, const QDict *qdict)
|
|
|
|
+{
|
|
|
|
+ Error *error = NULL;
|
|
|
|
+
|
|
|
|
+ int dir = qdict_get_try_bool(qdict, "directory", 0);
|
|
|
|
+ const char *backup_file = qdict_get_str(qdict, "backupfile");
|
|
|
|
+ const char *devlist = qdict_get_try_str(qdict, "devlist");
|
|
|
|
+ int64_t speed = qdict_get_try_int(qdict, "speed", 0);
|
|
|
|
+
|
|
|
|
+ qmp_backup(
|
|
|
|
+ backup_file,
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ NULL, // PBS password
|
|
|
|
+ NULL, // PBS keyfile
|
|
|
|
+ NULL, // PBS key_password
|
|
|
|
+ NULL, // PBS fingerprint
|
|
|
|
+ NULL, // PBS backup-id
|
2020-06-30 11:28:24 +03:00
|
|
|
+ false, 0, // PBS backup-time
|
2020-04-07 17:53:19 +03:00
|
|
|
+ true, dir ? BACKUP_FORMAT_DIR : BACKUP_FORMAT_VMA,
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ NULL, NULL,
|
2020-04-07 17:53:19 +03:00
|
|
|
+ devlist, qdict_haskey(qdict, "speed"), speed, &error);
|
|
|
|
+
|
|
|
|
+ hmp_handle_error(mon, error);
|
|
|
|
+}
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/blockdev.c b/blockdev.c
|
2023-05-15 16:39:54 +03:00
|
|
|
index 9a010f3a86..b9505c95d3 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/blockdev.c
|
|
|
|
+++ b/blockdev.c
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -37,6 +37,7 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
#include "block/blockjob.h"
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
#include "block/dirty-bitmap.h"
|
2020-03-10 17:12:50 +03:00
|
|
|
#include "block/qdict.h"
|
|
|
|
+#include "block/blockjob_int.h"
|
|
|
|
#include "block/throttle-groups.h"
|
|
|
|
#include "monitor/monitor.h"
|
|
|
|
#include "qemu/error-report.h"
|
|
|
|
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
index a166bff3d5..4b75966c2e 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/hmp-commands-info.hx
|
|
|
|
+++ b/hmp-commands-info.hx
|
2022-12-14 17:16:32 +03:00
|
|
|
@@ -486,6 +486,20 @@ SRST
|
2021-10-11 14:55:34 +03:00
|
|
|
Show the current VM UUID.
|
2020-04-07 17:53:19 +03:00
|
|
|
ERST
|
|
|
|
|
2021-10-11 14:55:34 +03:00
|
|
|
+
|
2020-03-10 17:12:50 +03:00
|
|
|
+ {
|
|
|
|
+ .name = "backup",
|
|
|
|
+ .args_type = "",
|
|
|
|
+ .params = "",
|
|
|
|
+ .help = "show backup status",
|
2020-04-07 17:53:19 +03:00
|
|
|
+ .cmd = hmp_info_backup,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ },
|
|
|
|
+
|
2020-04-07 17:53:19 +03:00
|
|
|
+SRST
|
|
|
|
+ ``info backup``
|
|
|
|
+ Show backup status.
|
|
|
|
+ERST
|
|
|
|
+
|
2020-03-10 17:12:50 +03:00
|
|
|
#if defined(CONFIG_SLIRP)
|
2020-04-07 17:53:19 +03:00
|
|
|
{
|
|
|
|
.name = "usernet",
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/hmp-commands.hx b/hmp-commands.hx
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
index b66d7fc4ab..9b6b8e2e9c 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/hmp-commands.hx
|
|
|
|
+++ b/hmp-commands.hx
|
update submodule and patches to 7.1.0
Notable changes:
* The only big change is the switch to using a custom QIOChannel for
savevm-async, because the previously used QEMUFileOps was dropped.
Changes to the current implementation:
* Switch to vector based methods as required for an IO channel. For
short reads the passed-in IO vector is stuffed with zeroes at the
end, just to be sure.
* For reading: The documentation in include/io/channel.h states that
at least one byte should be read, so also error out when whe are
at the very end instead of returning 0.
* For reading: Fix off-by-one error when request goes beyond end.
The wrong code piece was:
if ((pos + size) > maxlen) {
size = maxlen - pos - 1;
}
Previously, the last byte would not be read. It's actually
possible to get a snapshot .raw file that has content all the way
up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
trailing zero bytes (I wrote a script to do it).
Luckily, it didn't cause a real issue, because qemu_loadvm_state()
is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
section. The buffer for reading it is simply freed up afterwards
and the function will assume that it read the whole section, even
if that's not the case.
* For writing: Make use of the generated blk_pwritev() wrapper
instead of manually wrapping the coroutine to simplify and save a
few lines.
* Adapt to changed interfaces for blk_{pread,pwrite}:
* a9262f551e ("block: Change blk_{pread,pwrite}() param order")
* 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
* bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
Those changes especially affected the qemu-img dd patches, because
the context also changed, but also some of our block drivers used
the functions.
* Drop qemu-common.h include: it got renamed after essentially
everything was moved to other headers. The only remaining user I
could find for things dropped from the header between 7.0 and 7.1
was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
already includes the header to which the function was moved.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
|
|
|
@@ -101,6 +101,35 @@ ERST
|
2020-04-07 17:53:19 +03:00
|
|
|
SRST
|
|
|
|
``block_stream``
|
|
|
|
Copy data from a backing file into a block device.
|
|
|
|
+ERST
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ {
|
|
|
|
+ .name = "backup",
|
|
|
|
+ .args_type = "directory:-d,backupfile:s,speed:o?,devlist:s?",
|
|
|
|
+ .params = "[-d] backupfile [speed [devlist]]",
|
|
|
|
+ .help = "create a VM Backup."
|
|
|
|
+ "\n\t\t\t Use -d to dump data into a directory instead"
|
|
|
|
+ "\n\t\t\t of using VMA format.",
|
|
|
|
+ .cmd = hmp_backup,
|
|
|
|
+ },
|
|
|
|
+
|
2020-04-07 17:53:19 +03:00
|
|
|
+SRST
|
|
|
|
+``backup``
|
|
|
|
+ Create a VM backup.
|
|
|
|
+ERST
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ {
|
|
|
|
+ .name = "backup_cancel",
|
|
|
|
+ .args_type = "",
|
|
|
|
+ .params = "",
|
|
|
|
+ .help = "cancel the current VM backup",
|
2020-04-07 17:53:19 +03:00
|
|
|
+ .cmd = hmp_backup_cancel,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ },
|
|
|
|
+
|
2020-04-07 17:53:19 +03:00
|
|
|
+SRST
|
|
|
|
+``backup_cancel``
|
|
|
|
+ Cancel the current VM backup.
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2020-04-07 17:53:19 +03:00
|
|
|
ERST
|
2020-03-10 17:12:50 +03:00
|
|
|
|
|
|
|
{
|
|
|
|
diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
index c012bad741..2e504db706 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/include/monitor/hmp.h
|
|
|
|
+++ b/include/monitor/hmp.h
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -32,6 +32,7 @@ void hmp_info_savevm(Monitor *mon, const QDict *qdict);
|
2021-05-27 13:43:32 +03:00
|
|
|
void hmp_info_migrate(Monitor *mon, const QDict *qdict);
|
2020-03-10 17:12:50 +03:00
|
|
|
void hmp_info_migrate_capabilities(Monitor *mon, const QDict *qdict);
|
|
|
|
void hmp_info_migrate_parameters(Monitor *mon, const QDict *qdict);
|
|
|
|
+void hmp_info_backup(Monitor *mon, const QDict *qdict);
|
|
|
|
void hmp_info_cpus(Monitor *mon, const QDict *qdict);
|
2020-04-07 17:53:19 +03:00
|
|
|
void hmp_info_vnc(Monitor *mon, const QDict *qdict);
|
|
|
|
void hmp_info_spice(Monitor *mon, const QDict *qdict);
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -84,6 +85,8 @@ void hmp_change_vnc(Monitor *mon, const char *device, const char *target,
|
|
|
|
void hmp_change_medium(Monitor *mon, const char *device, const char *target,
|
|
|
|
const char *arg, const char *read_only, bool force,
|
|
|
|
Error **errp);
|
2020-03-10 17:12:50 +03:00
|
|
|
+void hmp_backup(Monitor *mon, const QDict *qdict);
|
|
|
|
+void hmp_backup_cancel(Monitor *mon, const QDict *qdict);
|
2020-04-07 17:53:19 +03:00
|
|
|
void hmp_migrate(Monitor *mon, const QDict *qdict);
|
|
|
|
void hmp_device_add(Monitor *mon, const QDict *qdict);
|
|
|
|
void hmp_device_del(Monitor *mon, const QDict *qdict);
|
2021-02-11 19:11:11 +03:00
|
|
|
diff --git a/meson.build b/meson.build
|
2023-05-15 16:39:54 +03:00
|
|
|
index 603cdb97bb..d307d8eabf 100644
|
2021-02-11 19:11:11 +03:00
|
|
|
--- a/meson.build
|
|
|
|
+++ b/meson.build
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -1528,6 +1528,7 @@ keyutils = dependency('libkeyutils', required: false,
|
2021-02-11 19:11:11 +03:00
|
|
|
has_gettid = cc.has_function('gettid')
|
|
|
|
|
|
|
|
libuuid = cc.find_library('uuid', required: true)
|
|
|
|
+libproxmox_backup_qemu = cc.find_library('proxmox_backup_qemu', required: true)
|
|
|
|
|
2022-02-11 12:24:33 +03:00
|
|
|
# libselinux
|
|
|
|
selinux = dependency('libselinux',
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
index 435f9334f9..9e1bd57aeb 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/monitor/hmp-cmds.c
|
|
|
|
+++ b/monitor/hmp-cmds.c
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -21,6 +21,7 @@
|
|
|
|
#include "qemu/help_option.h"
|
|
|
|
#include "monitor/monitor-internal.h"
|
|
|
|
#include "qapi/error.h"
|
|
|
|
+#include "qapi/qapi-commands-block-core.h"
|
|
|
|
#include "qapi/qapi-commands-control.h"
|
|
|
|
#include "qapi/qapi-commands-migration.h"
|
|
|
|
#include "qapi/qapi-commands-misc.h"
|
|
|
|
@@ -144,6 +145,50 @@ void hmp_sync_profile(Monitor *mon, const QDict *qdict)
|
|
|
|
}
|
2020-03-10 17:12:50 +03:00
|
|
|
}
|
|
|
|
|
|
|
|
+void hmp_info_backup(Monitor *mon, const QDict *qdict)
|
|
|
|
+{
|
|
|
|
+ BackupStatus *info;
|
|
|
|
+
|
|
|
|
+ info = qmp_query_backup(NULL);
|
|
|
|
+
|
|
|
|
+ if (!info) {
|
|
|
|
+ monitor_printf(mon, "Backup status: not initialized\n");
|
|
|
|
+ return;
|
|
|
|
+ }
|
|
|
|
+
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ if (info->status) {
|
|
|
|
+ if (info->errmsg) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ monitor_printf(mon, "Backup status: %s - %s\n",
|
|
|
|
+ info->status, info->errmsg);
|
|
|
|
+ } else {
|
|
|
|
+ monitor_printf(mon, "Backup status: %s\n", info->status);
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ if (info->backup_file) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ monitor_printf(mon, "Start time: %s", ctime(&info->start_time));
|
|
|
|
+ if (info->end_time) {
|
|
|
|
+ monitor_printf(mon, "End time: %s", ctime(&info->end_time));
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ int per = (info->has_total && info->total &&
|
|
|
|
+ info->has_transferred && info->transferred) ?
|
|
|
|
+ (info->transferred * 100)/info->total : 0;
|
|
|
|
+ int zero_per = (info->has_total && info->total &&
|
|
|
|
+ info->has_zero_bytes && info->zero_bytes) ?
|
|
|
|
+ (info->zero_bytes * 100)/info->total : 0;
|
|
|
|
+ monitor_printf(mon, "Backup file: %s\n", info->backup_file);
|
|
|
|
+ monitor_printf(mon, "Backup uuid: %s\n", info->uuid);
|
|
|
|
+ monitor_printf(mon, "Total size: %zd\n", info->total);
|
|
|
|
+ monitor_printf(mon, "Transferred bytes: %zd (%d%%)\n",
|
|
|
|
+ info->transferred, per);
|
|
|
|
+ monitor_printf(mon, "Zero bytes: %zd (%d%%)\n",
|
|
|
|
+ info->zero_bytes, zero_per);
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ qapi_free_BackupStatus(info);
|
|
|
|
+}
|
|
|
|
+
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
void hmp_exit_preconfig(Monitor *mon, const QDict *qdict)
|
2020-03-10 17:12:50 +03:00
|
|
|
{
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
Error *err = NULL;
|
2020-03-10 17:12:50 +03:00
|
|
|
diff --git a/proxmox-backup-client.c b/proxmox-backup-client.c
|
|
|
|
new file mode 100644
|
2020-10-29 16:10:36 +03:00
|
|
|
index 0000000000..a8f6653a81
|
2020-03-10 17:12:50 +03:00
|
|
|
--- /dev/null
|
|
|
|
+++ b/proxmox-backup-client.c
|
2020-10-29 16:10:36 +03:00
|
|
|
@@ -0,0 +1,176 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
+#include "proxmox-backup-client.h"
|
|
|
|
+#include "qemu/main-loop.h"
|
|
|
|
+#include "block/aio-wait.h"
|
|
|
|
+#include "qapi/error.h"
|
|
|
|
+
|
|
|
|
+/* Proxmox Backup Server client bindings using coroutines */
|
|
|
|
+
|
|
|
|
+typedef struct BlockOnCoroutineWrapper {
|
|
|
|
+ AioContext *ctx;
|
|
|
|
+ CoroutineEntry *entry;
|
|
|
|
+ void *entry_arg;
|
|
|
|
+ bool finished;
|
|
|
|
+} BlockOnCoroutineWrapper;
|
|
|
|
+
|
|
|
|
+static void coroutine_fn block_on_coroutine_wrapper(void *opaque)
|
|
|
|
+{
|
|
|
|
+ BlockOnCoroutineWrapper *wrapper = opaque;
|
|
|
|
+ wrapper->entry(wrapper->entry_arg);
|
|
|
|
+ wrapper->finished = true;
|
|
|
|
+ aio_wait_kick();
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+void block_on_coroutine_fn(CoroutineEntry *entry, void *entry_arg)
|
|
|
|
+{
|
|
|
|
+ assert(!qemu_in_coroutine());
|
|
|
|
+
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
|
|
|
+ BlockOnCoroutineWrapper wrapper = {
|
|
|
|
+ .finished = false,
|
|
|
|
+ .entry = entry,
|
|
|
|
+ .entry_arg = entry_arg,
|
|
|
|
+ .ctx = ctx,
|
|
|
|
+ };
|
|
|
|
+ Coroutine *wrapper_co = qemu_coroutine_create(block_on_coroutine_wrapper, &wrapper);
|
|
|
|
+ aio_co_enter(ctx, wrapper_co);
|
|
|
|
+ AIO_WAIT_WHILE(ctx, !wrapper.finished);
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+// This is called from another thread, so we use aio_co_schedule()
|
|
|
|
+static void proxmox_backup_schedule_wake(void *data) {
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData *waker = (CoCtxData *)data;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ aio_co_schedule(waker->ctx, waker->co);
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_connect(ProxmoxBackupHandle *pbs, Error **errp)
|
|
|
|
+{
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
+
|
|
|
|
+ proxmox_backup_connect_async(pbs, proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
+ if (errp) error_setg(errp, "backup connect failed: %s", pbs_err ? pbs_err : "unknown error");
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
+ }
|
|
|
|
+ return pbs_res;
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_add_config(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ const char *name,
|
|
|
|
+ const uint8_t *data,
|
|
|
|
+ uint64_t size,
|
|
|
|
+ Error **errp)
|
|
|
|
+{
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
+
|
|
|
|
+ proxmox_backup_add_config_async(
|
|
|
|
+ pbs, name, data, size ,proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
+ if (errp) error_setg(errp, "backup add_config %s failed: %s", name, pbs_err ? pbs_err : "unknown error");
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
+ }
|
|
|
|
+ return pbs_res;
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_register_image(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ const char *device_name,
|
|
|
|
+ uint64_t size,
|
|
|
|
+ Error **errp)
|
|
|
|
+{
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
+
|
|
|
|
+ proxmox_backup_register_image_async(
|
|
|
|
+ pbs, device_name, size ,proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
+ if (errp) error_setg(errp, "backup register image failed: %s", pbs_err ? pbs_err : "unknown error");
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
+ }
|
|
|
|
+ return pbs_res;
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_finish(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ Error **errp)
|
|
|
|
+{
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
+
|
|
|
|
+ proxmox_backup_finish_async(
|
|
|
|
+ pbs, proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
+ if (errp) error_setg(errp, "backup finish failed: %s", pbs_err ? pbs_err : "unknown error");
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
+ }
|
|
|
|
+ return pbs_res;
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_close_image(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ uint8_t dev_id,
|
|
|
|
+ Error **errp)
|
|
|
|
+{
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
+
|
|
|
|
+ proxmox_backup_close_image_async(
|
|
|
|
+ pbs, dev_id, proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
+ if (errp) error_setg(errp, "backup close image failed: %s", pbs_err ? pbs_err : "unknown error");
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
+ }
|
|
|
|
+ return pbs_res;
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_write_data(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ uint8_t dev_id,
|
|
|
|
+ const uint8_t *data,
|
|
|
|
+ uint64_t offset,
|
|
|
|
+ uint64_t size,
|
|
|
|
+ Error **errp)
|
|
|
|
+{
|
|
|
|
+ Coroutine *co = qemu_coroutine_self();
|
|
|
|
+ AioContext *ctx = qemu_get_current_aio_context();
|
2020-10-29 16:10:36 +03:00
|
|
|
+ CoCtxData waker = { .co = co, .ctx = ctx };
|
2020-03-10 17:12:50 +03:00
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
+
|
|
|
|
+ proxmox_backup_write_data_async(
|
|
|
|
+ pbs, dev_id, data, offset, size, proxmox_backup_schedule_wake, &waker, &pbs_res, &pbs_err);
|
|
|
|
+ qemu_coroutine_yield();
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
+ if (errp) error_setg(errp, "backup write data failed: %s", pbs_err ? pbs_err : "unknown error");
|
|
|
|
+ if (pbs_err) proxmox_backup_free_error(pbs_err);
|
|
|
|
+ }
|
|
|
|
+ return pbs_res;
|
|
|
|
+}
|
|
|
|
diff --git a/proxmox-backup-client.h b/proxmox-backup-client.h
|
|
|
|
new file mode 100644
|
2020-10-29 16:10:36 +03:00
|
|
|
index 0000000000..1dda8b7d8f
|
2020-03-10 17:12:50 +03:00
|
|
|
--- /dev/null
|
|
|
|
+++ b/proxmox-backup-client.h
|
2020-10-29 16:10:36 +03:00
|
|
|
@@ -0,0 +1,59 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
+#ifndef PROXMOX_BACKUP_CLIENT_H
|
|
|
|
+#define PROXMOX_BACKUP_CLIENT_H
|
|
|
|
+
|
|
|
|
+#include "qemu/osdep.h"
|
|
|
|
+#include "qemu/coroutine.h"
|
|
|
|
+#include "proxmox-backup-qemu.h"
|
|
|
|
+
|
2020-10-29 16:10:36 +03:00
|
|
|
+typedef struct CoCtxData {
|
|
|
|
+ Coroutine *co;
|
|
|
|
+ AioContext *ctx;
|
|
|
|
+ void *data;
|
|
|
|
+} CoCtxData;
|
|
|
|
+
|
|
|
|
+// FIXME: Remove once coroutines are supported for QMP
|
2020-03-10 17:12:50 +03:00
|
|
|
+void block_on_coroutine_fn(CoroutineEntry *entry, void *entry_arg);
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_connect(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ Error **errp);
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_add_config(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ const char *name,
|
|
|
|
+ const uint8_t *data,
|
|
|
|
+ uint64_t size,
|
|
|
|
+ Error **errp);
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_register_image(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ const char *device_name,
|
|
|
|
+ uint64_t size,
|
|
|
|
+ Error **errp);
|
|
|
|
+
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_finish(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ Error **errp);
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_close_image(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ uint8_t dev_id,
|
|
|
|
+ Error **errp);
|
|
|
|
+
|
|
|
|
+int coroutine_fn
|
|
|
|
+proxmox_backup_co_write_data(
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ uint8_t dev_id,
|
|
|
|
+ const uint8_t *data,
|
|
|
|
+ uint64_t offset,
|
|
|
|
+ uint64_t size,
|
|
|
|
+ Error **errp);
|
|
|
|
+
|
|
|
|
+
|
|
|
|
+#endif /* PROXMOX_BACKUP_CLIENT_H */
|
|
|
|
diff --git a/pve-backup.c b/pve-backup.c
|
|
|
|
new file mode 100644
|
2023-05-15 16:39:55 +03:00
|
|
|
index 0000000000..f77892a509
|
2020-03-10 17:12:50 +03:00
|
|
|
--- /dev/null
|
|
|
|
+++ b/pve-backup.c
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -0,0 +1,938 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
+#include "proxmox-backup-client.h"
|
|
|
|
+#include "vma.h"
|
|
|
|
+
|
|
|
|
+#include "qemu/osdep.h"
|
|
|
|
+#include "qemu/module.h"
|
|
|
|
+#include "sysemu/block-backend.h"
|
|
|
|
+#include "sysemu/blockdev.h"
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+#include "block/block_int-global-state.h"
|
2020-03-10 17:12:50 +03:00
|
|
|
+#include "block/blockjob.h"
|
|
|
|
+#include "qapi/qapi-commands-block.h"
|
|
|
|
+#include "qapi/qmp/qerror.h"
|
|
|
|
+
|
|
|
|
+/* PVE backup state and related function */
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+/*
|
|
|
|
+ * Note: A resume from a qemu_coroutine_yield can happen in a different thread,
|
|
|
|
+ * so you may not use normal mutexes within coroutines:
|
|
|
|
+ *
|
|
|
|
+ * ---bad-example---
|
|
|
|
+ * qemu_rec_mutex_lock(lock)
|
|
|
|
+ * ...
|
|
|
|
+ * qemu_coroutine_yield() // wait for something
|
|
|
|
+ * // we are now inside a different thread
|
|
|
|
+ * qemu_rec_mutex_unlock(lock) // Crash - wrong thread!!
|
|
|
|
+ * ---end-bad-example--
|
|
|
|
+ *
|
|
|
|
+ * ==> Always use CoMutext inside coroutines.
|
|
|
|
+ * ==> Never acquire/release AioContext withing coroutines (because that use QemuRecMutex)
|
|
|
|
+ *
|
|
|
|
+ */
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+static struct PVEBackupState {
|
|
|
|
+ struct {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ // Everithing accessed from qmp_backup_query command is protected using lock
|
|
|
|
+ QemuMutex lock;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ Error *error;
|
|
|
|
+ time_t start_time;
|
|
|
|
+ time_t end_time;
|
|
|
|
+ char *backup_file;
|
|
|
|
+ uuid_t uuid;
|
|
|
|
+ char uuid_str[37];
|
|
|
|
+ size_t total;
|
|
|
|
+ size_t transferred;
|
|
|
|
+ size_t zero_bytes;
|
|
|
|
+ } stat;
|
|
|
|
+ int64_t speed;
|
|
|
|
+ VmaWriter *vmaw;
|
|
|
|
+ ProxmoxBackupHandle *pbs;
|
|
|
|
+ GList *di_list;
|
2021-03-03 12:56:02 +03:00
|
|
|
+ QemuMutex backup_mutex;
|
|
|
|
+ CoMutex dump_callback_mutex;
|
2020-03-10 17:12:50 +03:00
|
|
|
+} backup_state;
|
|
|
|
+
|
|
|
|
+static void pvebackup_init(void)
|
|
|
|
+{
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_init(&backup_state.stat.lock);
|
|
|
|
+ qemu_mutex_init(&backup_state.backup_mutex);
|
|
|
|
+ qemu_co_mutex_init(&backup_state.dump_callback_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+}
|
|
|
|
+
|
|
|
|
+// initialize PVEBackupState at startup
|
|
|
|
+opts_init(pvebackup_init);
|
|
|
|
+
|
|
|
|
+typedef struct PVEBackupDevInfo {
|
|
|
|
+ BlockDriverState *bs;
|
|
|
|
+ size_t size;
|
|
|
|
+ uint8_t dev_id;
|
|
|
|
+ bool completed;
|
|
|
|
+ char targetfile[PATH_MAX];
|
|
|
|
+ BlockDriverState *target;
|
|
|
|
+} PVEBackupDevInfo;
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+static void pvebackup_run_next_job(void);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+static BlockJob *
|
|
|
|
+lookup_active_block_job(PVEBackupDevInfo *di)
|
|
|
|
+{
|
|
|
|
+ if (!di->completed && di->bs) {
|
2022-12-14 17:16:32 +03:00
|
|
|
+ WITH_JOB_LOCK_GUARD() {
|
|
|
|
+ for (BlockJob *job = block_job_next_locked(NULL); job; job = block_job_next_locked(job)) {
|
|
|
|
+ if (job->job.driver->job_type != JOB_TYPE_BACKUP) {
|
|
|
|
+ continue;
|
|
|
|
+ }
|
2021-03-03 12:56:02 +03:00
|
|
|
+
|
2022-12-14 17:16:32 +03:00
|
|
|
+ BackupBlockJob *bjob = container_of(job, BackupBlockJob, common);
|
|
|
|
+ if (bjob && bjob->source_bs == di->bs) {
|
|
|
|
+ return job;
|
|
|
|
+ }
|
2021-03-03 12:56:02 +03:00
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+ return NULL;
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+static void pvebackup_propagate_error(Error *err)
|
|
|
|
+{
|
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
+ error_propagate(&backup_state.stat.error, err);
|
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+static bool pvebackup_error_or_canceled(void)
|
|
|
|
+{
|
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
+ bool error_or_canceled = !!backup_state.stat.error;
|
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
+
|
|
|
|
+ return error_or_canceled;
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+static void pvebackup_add_transfered_bytes(size_t transferred, size_t zero_bytes)
|
|
|
|
+{
|
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
|
|
|
+ backup_state.stat.zero_bytes += zero_bytes;
|
|
|
|
+ backup_state.stat.transferred += transferred;
|
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+// This may get called from multiple coroutines in multiple io-threads
|
|
|
|
+// Note1: this may get called after job_cancel()
|
2020-03-10 17:12:50 +03:00
|
|
|
+static int coroutine_fn
|
2021-03-03 12:56:02 +03:00
|
|
|
+pvebackup_co_dump_pbs_cb(
|
2020-03-10 17:12:50 +03:00
|
|
|
+ void *opaque,
|
|
|
|
+ uint64_t start,
|
|
|
|
+ uint64_t bytes,
|
|
|
|
+ const void *pbuf)
|
|
|
|
+{
|
|
|
|
+ assert(qemu_in_coroutine());
|
|
|
|
+
|
|
|
|
+ const uint64_t size = bytes;
|
|
|
|
+ const unsigned char *buf = pbuf;
|
|
|
|
+ PVEBackupDevInfo *di = opaque;
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ assert(backup_state.pbs);
|
|
|
|
+
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
+ int pbs_res = -1;
|
|
|
|
+
|
|
|
|
+ qemu_co_mutex_lock(&backup_state.dump_callback_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ // avoid deadlock if job is cancelled
|
|
|
|
+ if (pvebackup_error_or_canceled()) {
|
|
|
|
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
|
|
|
|
+ return -1;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ pbs_res = proxmox_backup_co_write_data(backup_state.pbs, di->dev_id, buf, start, size, &local_err);
|
|
|
|
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
|
|
|
|
+
|
|
|
|
+ if (pbs_res < 0) {
|
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
+ return pbs_res;
|
|
|
|
+ } else {
|
|
|
|
+ pvebackup_add_transfered_bytes(size, !buf ? size : 0);
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ return size;
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+// This may get called from multiple coroutines in multiple io-threads
|
|
|
|
+static int coroutine_fn
|
|
|
|
+pvebackup_co_dump_vma_cb(
|
|
|
|
+ void *opaque,
|
|
|
|
+ uint64_t start,
|
|
|
|
+ uint64_t bytes,
|
|
|
|
+ const void *pbuf)
|
|
|
|
+{
|
|
|
|
+ assert(qemu_in_coroutine());
|
|
|
|
+
|
|
|
|
+ const uint64_t size = bytes;
|
|
|
|
+ const unsigned char *buf = pbuf;
|
|
|
|
+ PVEBackupDevInfo *di = opaque;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ int ret = -1;
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ assert(backup_state.vmaw);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ uint64_t remaining = size;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ uint64_t cluster_num = start / VMA_CLUSTER_SIZE;
|
|
|
|
+ if ((cluster_num * VMA_CLUSTER_SIZE) != start) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ Error *local_err = NULL;
|
2021-03-03 12:56:02 +03:00
|
|
|
+ error_setg(&local_err,
|
|
|
|
+ "got unaligned write inside backup dump "
|
|
|
|
+ "callback (sector %ld)", start);
|
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
+ return -1; // not aligned to cluster size
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ while (remaining > 0) {
|
|
|
|
+ qemu_co_mutex_lock(&backup_state.dump_callback_mutex);
|
|
|
|
+ // avoid deadlock if job is cancelled
|
|
|
|
+ if (pvebackup_error_or_canceled()) {
|
|
|
|
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
|
|
|
|
+ return -1;
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ size_t zero_bytes = 0;
|
|
|
|
+ ret = vma_writer_write(backup_state.vmaw, di->dev_id, cluster_num, buf, &zero_bytes);
|
|
|
|
+ qemu_co_mutex_unlock(&backup_state.dump_callback_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ ++cluster_num;
|
|
|
|
+ if (buf) {
|
|
|
|
+ buf += VMA_CLUSTER_SIZE;
|
|
|
|
+ }
|
|
|
|
+ if (ret < 0) {
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
+ vma_writer_error_propagate(backup_state.vmaw, &local_err);
|
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
+ return ret;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ } else {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ if (remaining >= VMA_CLUSTER_SIZE) {
|
|
|
|
+ assert(ret == VMA_CLUSTER_SIZE);
|
|
|
|
+ pvebackup_add_transfered_bytes(VMA_CLUSTER_SIZE, zero_bytes);
|
|
|
|
+ remaining -= VMA_CLUSTER_SIZE;
|
|
|
|
+ } else {
|
|
|
|
+ assert(ret == remaining);
|
|
|
|
+ pvebackup_add_transfered_bytes(remaining, zero_bytes);
|
|
|
|
+ remaining = 0;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ return size;
|
|
|
|
+}
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+// assumes the caller holds backup_mutex
|
|
|
|
+static void coroutine_fn pvebackup_co_cleanup(void *unused)
|
2020-03-10 17:12:50 +03:00
|
|
|
+{
|
|
|
|
+ assert(qemu_in_coroutine());
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ backup_state.stat.end_time = time(NULL);
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ if (backup_state.vmaw) {
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
+ vma_writer_close(backup_state.vmaw, &local_err);
|
|
|
|
+
|
|
|
|
+ if (local_err != NULL) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ backup_state.vmaw = NULL;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (backup_state.pbs) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ if (!pvebackup_error_or_canceled()) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ Error *local_err = NULL;
|
|
|
|
+ proxmox_backup_co_finish(backup_state.pbs, &local_err);
|
|
|
|
+ if (local_err != NULL) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ pvebackup_propagate_error(local_err);
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ proxmox_backup_disconnect(backup_state.pbs);
|
|
|
|
+ backup_state.pbs = NULL;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ g_list_free(backup_state.di_list);
|
|
|
|
+ backup_state.di_list = NULL;
|
|
|
|
+}
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+// assumes the caller holds backup_mutex
|
|
|
|
+static void coroutine_fn pvebackup_complete_stream(void *opaque)
|
2020-03-10 17:12:50 +03:00
|
|
|
+{
|
2021-03-03 12:56:02 +03:00
|
|
|
+ PVEBackupDevInfo *di = opaque;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ bool error_or_canceled = pvebackup_error_or_canceled();
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ if (backup_state.vmaw) {
|
|
|
|
+ vma_writer_close_stream(backup_state.vmaw, di->dev_id);
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (backup_state.pbs && !error_or_canceled) {
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
+ proxmox_backup_co_close_image(backup_state.pbs, di->dev_id, &local_err);
|
|
|
|
+ if (local_err != NULL) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ pvebackup_propagate_error(local_err);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+ }
|
2021-03-03 12:56:02 +03:00
|
|
|
+}
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+static void pvebackup_complete_cb(void *opaque, int ret)
|
|
|
|
+{
|
|
|
|
+ assert(!qemu_in_coroutine());
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ PVEBackupDevInfo *di = opaque;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_lock(&backup_state.backup_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ di->completed = true;
|
|
|
|
+
|
|
|
|
+ if (ret < 0) {
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
+ error_setg(&local_err, "job failed with err %d - %s", ret, strerror(-ret));
|
|
|
|
+ pvebackup_propagate_error(local_err);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ di->bs = NULL;
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ assert(di->target == NULL);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ block_on_coroutine_fn(pvebackup_complete_stream, di);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ // remove self from job queue
|
|
|
|
+ backup_state.di_list = g_list_remove(backup_state.di_list, di);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ g_free(di);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.backup_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ pvebackup_run_next_job();
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+static void pvebackup_cancel(void)
|
|
|
|
+{
|
|
|
|
+ assert(!qemu_in_coroutine());
|
|
|
|
+
|
|
|
|
+ Error *cancel_err = NULL;
|
|
|
|
+ error_setg(&cancel_err, "backup canceled");
|
|
|
|
+ pvebackup_propagate_error(cancel_err);
|
|
|
|
+
|
|
|
|
+ qemu_mutex_lock(&backup_state.backup_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ if (backup_state.vmaw) {
|
|
|
|
+ /* make sure vma writer does not block anymore */
|
2021-03-03 12:56:02 +03:00
|
|
|
+ vma_writer_set_error(backup_state.vmaw, "backup canceled");
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (backup_state.pbs) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ proxmox_backup_abort(backup_state.pbs, "backup canceled");
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.backup_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ for(;;) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ BlockJob *next_job = NULL;
|
|
|
|
+
|
|
|
|
+ qemu_mutex_lock(&backup_state.backup_mutex);
|
|
|
|
+
|
|
|
|
+ GList *l = backup_state.di_list;
|
|
|
|
+ while (l) {
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
+
|
|
|
|
+ BlockJob *job = lookup_active_block_job(di);
|
|
|
|
+ if (job != NULL) {
|
|
|
|
+ next_job = job;
|
|
|
|
+ break;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.backup_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ if (next_job) {
|
2022-02-11 12:24:33 +03:00
|
|
|
+ job_cancel_sync(&next_job->job, true);
|
2021-03-03 12:56:02 +03:00
|
|
|
+ } else {
|
|
|
|
+ break;
|
|
|
|
+ }
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+}
|
|
|
|
+
|
|
|
|
+void qmp_backup_cancel(Error **errp)
|
|
|
|
+{
|
2021-03-03 12:56:02 +03:00
|
|
|
+ pvebackup_cancel();
|
2020-03-10 17:12:50 +03:00
|
|
|
+}
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+// assumes the caller holds backup_mutex
|
2020-03-10 17:12:50 +03:00
|
|
|
+static int coroutine_fn pvebackup_co_add_config(
|
|
|
|
+ const char *file,
|
|
|
|
+ const char *name,
|
|
|
|
+ BackupFormat format,
|
|
|
|
+ const char *backup_dir,
|
|
|
|
+ VmaWriter *vmaw,
|
|
|
|
+ ProxmoxBackupHandle *pbs,
|
|
|
|
+ Error **errp)
|
|
|
|
+{
|
|
|
|
+ int res = 0;
|
|
|
|
+
|
|
|
|
+ char *cdata = NULL;
|
|
|
|
+ gsize clen = 0;
|
|
|
|
+ GError *err = NULL;
|
|
|
|
+ if (!g_file_get_contents(file, &cdata, &clen, &err)) {
|
|
|
|
+ error_setg(errp, "unable to read file '%s'", file);
|
|
|
|
+ return 1;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ char *basename = g_path_get_basename(file);
|
|
|
|
+ if (name == NULL) name = basename;
|
|
|
|
+
|
|
|
|
+ if (format == BACKUP_FORMAT_VMA) {
|
|
|
|
+ if (vma_writer_add_config(vmaw, name, cdata, clen) != 0) {
|
|
|
|
+ error_setg(errp, "unable to add %s config data to vma archive", file);
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ } else if (format == BACKUP_FORMAT_PBS) {
|
|
|
|
+ if (proxmox_backup_co_add_config(pbs, name, (unsigned char *)cdata, clen, errp) < 0)
|
|
|
|
+ goto err;
|
|
|
|
+ } else if (format == BACKUP_FORMAT_DIR) {
|
|
|
|
+ char config_path[PATH_MAX];
|
|
|
|
+ snprintf(config_path, PATH_MAX, "%s/%s", backup_dir, name);
|
|
|
|
+ if (!g_file_set_contents(config_path, cdata, clen, &err)) {
|
|
|
|
+ error_setg(errp, "unable to write config file '%s'", config_path);
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ out:
|
|
|
|
+ g_free(basename);
|
|
|
|
+ g_free(cdata);
|
|
|
|
+ return res;
|
|
|
|
+
|
|
|
|
+ err:
|
|
|
|
+ res = -1;
|
|
|
|
+ goto out;
|
|
|
|
+}
|
|
|
|
+
|
2022-12-14 17:16:32 +03:00
|
|
|
+bool job_should_pause_locked(Job *job);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+static void pvebackup_run_next_job(void)
|
2020-03-10 17:12:50 +03:00
|
|
|
+{
|
2021-03-03 12:56:02 +03:00
|
|
|
+ assert(!qemu_in_coroutine());
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_lock(&backup_state.backup_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ GList *l = backup_state.di_list;
|
|
|
|
+ while (l) {
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ BlockJob *job = lookup_active_block_job(di);
|
|
|
|
+
|
|
|
|
+ if (job) {
|
|
|
|
+ qemu_mutex_unlock(&backup_state.backup_mutex);
|
|
|
|
+
|
2022-12-14 17:16:32 +03:00
|
|
|
+ WITH_JOB_LOCK_GUARD() {
|
|
|
|
+ if (job_should_pause_locked(&job->job)) {
|
|
|
|
+ bool error_or_canceled = pvebackup_error_or_canceled();
|
|
|
|
+ if (error_or_canceled) {
|
|
|
|
+ job_cancel_sync_locked(&job->job, true);
|
|
|
|
+ } else {
|
|
|
|
+ job_resume_locked(&job->job);
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+ }
|
2021-03-03 12:56:02 +03:00
|
|
|
+ return;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+ }
|
2021-03-03 12:56:02 +03:00
|
|
|
+
|
|
|
|
+ block_on_coroutine_fn(pvebackup_co_cleanup, NULL); // no more jobs, run cleanup
|
|
|
|
+
|
|
|
|
+ qemu_mutex_unlock(&backup_state.backup_mutex);
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+static bool create_backup_jobs(void) {
|
|
|
|
+
|
|
|
|
+ assert(!qemu_in_coroutine());
|
|
|
|
+
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
+
|
2021-05-27 13:43:32 +03:00
|
|
|
+ BackupPerf perf = { .max_workers = 16 };
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ /* create and start all jobs (paused state) */
|
|
|
|
+ GList *l = backup_state.di_list;
|
|
|
|
+ while (l) {
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
+
|
|
|
|
+ assert(di->target != NULL);
|
|
|
|
+
|
|
|
|
+ AioContext *aio_context = bdrv_get_aio_context(di->bs);
|
|
|
|
+ aio_context_acquire(aio_context);
|
|
|
|
+
|
|
|
|
+ BlockJob *job = backup_job_create(
|
|
|
|
+ NULL, di->bs, di->target, backup_state.speed, MIRROR_SYNC_MODE_FULL, NULL,
|
2021-05-27 13:43:32 +03:00
|
|
|
+ BITMAP_SYNC_MODE_NEVER, false, NULL, &perf, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT,
|
|
|
|
+ JOB_DEFAULT, pvebackup_complete_cb, di, NULL, &local_err);
|
2021-03-03 12:56:02 +03:00
|
|
|
+
|
|
|
|
+ aio_context_release(aio_context);
|
|
|
|
+
|
|
|
|
+ if (!job || local_err != NULL) {
|
|
|
|
+ Error *create_job_err = NULL;
|
|
|
|
+ error_setg(&create_job_err, "backup_job_create failed: %s",
|
|
|
|
+ local_err ? error_get_pretty(local_err) : "null");
|
|
|
|
+
|
|
|
|
+ pvebackup_propagate_error(create_job_err);
|
|
|
|
+ break;
|
|
|
|
+ }
|
|
|
|
+ job_start(&job->job);
|
|
|
|
+
|
|
|
|
+ bdrv_unref(di->target);
|
|
|
|
+ di->target = NULL;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ bool errors = pvebackup_error_or_canceled();
|
|
|
|
+
|
|
|
|
+ if (errors) {
|
|
|
|
+ l = backup_state.di_list;
|
|
|
|
+ while (l) {
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
+
|
|
|
|
+ if (di->target) {
|
|
|
|
+ bdrv_unref(di->target);
|
|
|
|
+ di->target = NULL;
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ return errors;
|
2020-03-10 17:12:50 +03:00
|
|
|
+}
|
|
|
|
+
|
|
|
|
+typedef struct QmpBackupTask {
|
|
|
|
+ const char *backup_file;
|
|
|
|
+ const char *password;
|
|
|
|
+ const char *keyfile;
|
|
|
|
+ const char *key_password;
|
|
|
|
+ const char *backup_id;
|
|
|
|
+ bool has_backup_time;
|
|
|
|
+ const char *fingerprint;
|
|
|
|
+ int64_t backup_time;
|
|
|
|
+ bool has_format;
|
|
|
|
+ BackupFormat format;
|
|
|
|
+ const char *config_file;
|
|
|
|
+ const char *firewall_file;
|
|
|
|
+ const char *devlist;
|
|
|
|
+ bool has_speed;
|
|
|
|
+ int64_t speed;
|
|
|
|
+ Error **errp;
|
|
|
|
+ UuidInfo *result;
|
|
|
|
+} QmpBackupTask;
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+// assumes the caller holds backup_mutex
|
|
|
|
+static void coroutine_fn pvebackup_co_prepare(void *opaque)
|
2020-03-10 17:12:50 +03:00
|
|
|
+{
|
|
|
|
+ assert(qemu_in_coroutine());
|
|
|
|
+
|
|
|
|
+ QmpBackupTask *task = opaque;
|
|
|
|
+
|
|
|
|
+ task->result = NULL; // just to be sure
|
|
|
|
+
|
|
|
|
+ BlockBackend *blk;
|
|
|
|
+ BlockDriverState *bs = NULL;
|
|
|
|
+ const char *backup_dir = NULL;
|
|
|
|
+ Error *local_err = NULL;
|
|
|
|
+ uuid_t uuid;
|
|
|
|
+ VmaWriter *vmaw = NULL;
|
|
|
|
+ ProxmoxBackupHandle *pbs = NULL;
|
|
|
|
+ gchar **devs = NULL;
|
|
|
|
+ GList *di_list = NULL;
|
|
|
|
+ GList *l;
|
|
|
|
+ UuidInfo *uuid_info;
|
|
|
|
+
|
|
|
|
+ const char *config_name = "qemu-server.conf";
|
|
|
|
+ const char *firewall_name = "qemu-server.fw";
|
|
|
|
+
|
|
|
|
+ if (backup_state.di_list) {
|
2021-03-03 12:56:02 +03:00
|
|
|
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ "previous backup not finished");
|
|
|
|
+ return;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ /* Todo: try to auto-detect format based on file name */
|
|
|
|
+ BackupFormat format = task->has_format ? task->format : BACKUP_FORMAT_VMA;
|
|
|
|
+
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ if (task->devlist) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ devs = g_strsplit_set(task->devlist, ",;:", -1);
|
|
|
|
+
|
|
|
|
+ gchar **d = devs;
|
|
|
|
+ while (d && *d) {
|
|
|
|
+ blk = blk_by_name(*d);
|
|
|
|
+ if (blk) {
|
|
|
|
+ bs = blk_bs(blk);
|
|
|
|
+ if (!bdrv_is_inserted(bs)) {
|
|
|
|
+ error_setg(task->errp, QERR_DEVICE_HAS_NO_MEDIUM, *d);
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
|
|
|
|
+ di->bs = bs;
|
|
|
|
+ di_list = g_list_append(di_list, di);
|
|
|
|
+ } else {
|
|
|
|
+ error_set(task->errp, ERROR_CLASS_DEVICE_NOT_FOUND,
|
|
|
|
+ "Device '%s' not found", *d);
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ d++;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ } else {
|
|
|
|
+ BdrvNextIterator it;
|
|
|
|
+
|
|
|
|
+ bs = NULL;
|
|
|
|
+ for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
|
|
|
|
+ if (!bdrv_is_inserted(bs) || bdrv_is_read_only(bs)) {
|
|
|
|
+ continue;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
|
|
|
|
+ di->bs = bs;
|
|
|
|
+ di_list = g_list_append(di_list, di);
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (!di_list) {
|
|
|
|
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "empty device list");
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ size_t total = 0;
|
|
|
|
+
|
|
|
|
+ l = di_list;
|
|
|
|
+ while (l) {
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
+ if (bdrv_op_is_blocked(di->bs, BLOCK_OP_TYPE_BACKUP_SOURCE, task->errp)) {
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ ssize_t size = bdrv_getlength(di->bs);
|
|
|
|
+ if (size < 0) {
|
2023-01-30 13:20:42 +03:00
|
|
|
+ error_setg_errno(task->errp, -size, "bdrv_getlength failed");
|
2020-03-10 17:12:50 +03:00
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ di->size = size;
|
|
|
|
+ total += size;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ uuid_generate(uuid);
|
|
|
|
+
|
|
|
|
+ if (format == BACKUP_FORMAT_PBS) {
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ if (!task->password) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'password'");
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ if (!task->backup_id) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-id'");
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ if (!task->has_backup_time) {
|
|
|
|
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-time'");
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ int dump_cb_block_size = PROXMOX_BACKUP_DEFAULT_CHUNK_SIZE; // Hardcoded (4M)
|
|
|
|
+ firewall_name = "fw.conf";
|
|
|
|
+
|
|
|
|
+ char *pbs_err = NULL;
|
|
|
|
+ pbs = proxmox_backup_new(
|
|
|
|
+ task->backup_file,
|
|
|
|
+ task->backup_id,
|
|
|
|
+ task->backup_time,
|
|
|
|
+ dump_cb_block_size,
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ task->password,
|
|
|
|
+ task->keyfile,
|
|
|
|
+ task->key_password,
|
|
|
|
+ task->fingerprint,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ &pbs_err);
|
|
|
|
+
|
|
|
|
+ if (!pbs) {
|
|
|
|
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
|
|
|
|
+ "proxmox_backup_new failed: %s", pbs_err);
|
|
|
|
+ proxmox_backup_free_error(pbs_err);
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (proxmox_backup_co_connect(pbs, task->errp) < 0)
|
|
|
|
+ goto err;
|
|
|
|
+
|
|
|
|
+ /* register all devices */
|
|
|
|
+ l = di_list;
|
|
|
|
+ while (l) {
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
+
|
|
|
|
+ const char *devname = bdrv_get_device_name(di->bs);
|
|
|
|
+
|
|
|
|
+ int dev_id = proxmox_backup_co_register_image(pbs, devname, di->size, task->errp);
|
|
|
|
+ if (dev_id < 0)
|
|
|
|
+ goto err;
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ if (!(di->target = bdrv_backup_dump_create(dump_cb_block_size, di->size, pvebackup_co_dump_pbs_cb, di, task->errp))) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ di->dev_id = dev_id;
|
|
|
|
+ }
|
|
|
|
+ } else if (format == BACKUP_FORMAT_VMA) {
|
|
|
|
+ vmaw = vma_writer_create(task->backup_file, uuid, &local_err);
|
|
|
|
+ if (!vmaw) {
|
|
|
|
+ if (local_err) {
|
|
|
|
+ error_propagate(task->errp, local_err);
|
|
|
|
+ }
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ /* register all devices for vma writer */
|
|
|
|
+ l = di_list;
|
|
|
|
+ while (l) {
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ if (!(di->target = bdrv_backup_dump_create(VMA_CLUSTER_SIZE, di->size, pvebackup_co_dump_vma_cb, di, task->errp))) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ const char *devname = bdrv_get_device_name(di->bs);
|
|
|
|
+ di->dev_id = vma_writer_register_stream(vmaw, devname, di->size);
|
|
|
|
+ if (di->dev_id <= 0) {
|
|
|
|
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
|
|
|
|
+ "register_stream failed");
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+ } else if (format == BACKUP_FORMAT_DIR) {
|
|
|
|
+ if (mkdir(task->backup_file, 0640) != 0) {
|
|
|
|
+ error_setg_errno(task->errp, errno, "can't create directory '%s'\n",
|
|
|
|
+ task->backup_file);
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ backup_dir = task->backup_file;
|
|
|
|
+
|
|
|
|
+ l = di_list;
|
|
|
|
+ while (l) {
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
+
|
|
|
|
+ const char *devname = bdrv_get_device_name(di->bs);
|
|
|
|
+ snprintf(di->targetfile, PATH_MAX, "%s/%s.raw", backup_dir, devname);
|
|
|
|
+
|
|
|
|
+ int flags = BDRV_O_RDWR;
|
|
|
|
+ bdrv_img_create(di->targetfile, "raw", NULL, NULL, NULL,
|
|
|
|
+ di->size, flags, false, &local_err);
|
|
|
|
+ if (local_err) {
|
|
|
|
+ error_propagate(task->errp, local_err);
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+
|
2023-05-15 16:39:55 +03:00
|
|
|
+ di->target = bdrv_co_open(di->targetfile, NULL, NULL, flags, &local_err);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ if (!di->target) {
|
|
|
|
+ error_propagate(task->errp, local_err);
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+ } else {
|
|
|
|
+ error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "unknown backup format");
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+
|
|
|
|
+ /* add configuration file to archive */
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ if (task->config_file) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ if (pvebackup_co_add_config(task->config_file, config_name, format, backup_dir,
|
|
|
|
+ vmaw, pbs, task->errp) != 0) {
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ /* add firewall file to archive */
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ if (task->firewall_file) {
|
2020-03-10 17:12:50 +03:00
|
|
|
+ if (pvebackup_co_add_config(task->firewall_file, firewall_name, format, backup_dir,
|
|
|
|
+ vmaw, pbs, task->errp) != 0) {
|
|
|
|
+ goto err;
|
|
|
|
+ }
|
|
|
|
+ }
|
|
|
|
+ /* initialize global backup_state now */
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ if (backup_state.stat.error) {
|
|
|
|
+ error_free(backup_state.stat.error);
|
|
|
|
+ backup_state.stat.error = NULL;
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ backup_state.stat.start_time = time(NULL);
|
|
|
|
+ backup_state.stat.end_time = 0;
|
|
|
|
+
|
|
|
|
+ if (backup_state.stat.backup_file) {
|
|
|
|
+ g_free(backup_state.stat.backup_file);
|
|
|
|
+ }
|
|
|
|
+ backup_state.stat.backup_file = g_strdup(task->backup_file);
|
|
|
|
+
|
|
|
|
+ uuid_copy(backup_state.stat.uuid, uuid);
|
|
|
|
+ uuid_unparse_lower(uuid, backup_state.stat.uuid_str);
|
|
|
|
+ char *uuid_str = g_strdup(backup_state.stat.uuid_str);
|
|
|
|
+
|
|
|
|
+ backup_state.stat.total = total;
|
|
|
|
+ backup_state.stat.transferred = 0;
|
|
|
|
+ backup_state.stat.zero_bytes = 0;
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ backup_state.speed = (task->has_speed && task->speed > 0) ? task->speed : 0;
|
|
|
|
+
|
|
|
|
+ backup_state.vmaw = vmaw;
|
|
|
|
+ backup_state.pbs = pbs;
|
|
|
|
+
|
|
|
|
+ backup_state.di_list = di_list;
|
|
|
|
+
|
|
|
|
+ uuid_info = g_malloc0(sizeof(*uuid_info));
|
|
|
|
+ uuid_info->UUID = uuid_str;
|
|
|
|
+
|
|
|
|
+ task->result = uuid_info;
|
|
|
|
+ return;
|
|
|
|
+
|
|
|
|
+err:
|
|
|
|
+
|
|
|
|
+ l = di_list;
|
|
|
|
+ while (l) {
|
|
|
|
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
|
|
|
|
+ l = g_list_next(l);
|
|
|
|
+
|
|
|
|
+ if (di->target) {
|
2023-05-15 16:39:54 +03:00
|
|
|
+ bdrv_co_unref(di->target);
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (di->targetfile[0]) {
|
|
|
|
+ unlink(di->targetfile);
|
|
|
|
+ }
|
|
|
|
+ g_free(di);
|
|
|
|
+ }
|
|
|
|
+ g_list_free(di_list);
|
|
|
|
+
|
|
|
|
+ if (devs) {
|
|
|
|
+ g_strfreev(devs);
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (vmaw) {
|
|
|
|
+ Error *err = NULL;
|
|
|
|
+ vma_writer_close(vmaw, &err);
|
|
|
|
+ unlink(task->backup_file);
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (pbs) {
|
|
|
|
+ proxmox_backup_disconnect(pbs);
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ if (backup_dir) {
|
|
|
|
+ rmdir(backup_dir);
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ task->result = NULL;
|
|
|
|
+ return;
|
|
|
|
+}
|
|
|
|
+
|
|
|
|
+UuidInfo *qmp_backup(
|
|
|
|
+ const char *backup_file,
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ const char *password,
|
|
|
|
+ const char *keyfile,
|
|
|
|
+ const char *key_password,
|
|
|
|
+ const char *fingerprint,
|
|
|
|
+ const char *backup_id,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ bool has_backup_time, int64_t backup_time,
|
|
|
|
+ bool has_format, BackupFormat format,
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
+ const char *config_file,
|
|
|
|
+ const char *firewall_file,
|
|
|
|
+ const char *devlist,
|
2020-03-10 17:12:50 +03:00
|
|
|
+ bool has_speed, int64_t speed, Error **errp)
|
|
|
|
+{
|
|
|
|
+ QmpBackupTask task = {
|
|
|
|
+ .backup_file = backup_file,
|
|
|
|
+ .password = password,
|
|
|
|
+ .key_password = key_password,
|
|
|
|
+ .fingerprint = fingerprint,
|
|
|
|
+ .backup_id = backup_id,
|
|
|
|
+ .has_backup_time = has_backup_time,
|
|
|
|
+ .backup_time = backup_time,
|
|
|
|
+ .has_format = has_format,
|
|
|
|
+ .format = format,
|
|
|
|
+ .config_file = config_file,
|
|
|
|
+ .firewall_file = firewall_file,
|
|
|
|
+ .devlist = devlist,
|
|
|
|
+ .has_speed = has_speed,
|
|
|
|
+ .speed = speed,
|
|
|
|
+ .errp = errp,
|
|
|
|
+ };
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_lock(&backup_state.backup_mutex);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ block_on_coroutine_fn(pvebackup_co_prepare, &task);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ if (*errp == NULL) {
|
|
|
|
+ create_backup_jobs();
|
|
|
|
+ qemu_mutex_unlock(&backup_state.backup_mutex);
|
|
|
|
+ pvebackup_run_next_job();
|
|
|
|
+ } else {
|
|
|
|
+ qemu_mutex_unlock(&backup_state.backup_mutex);
|
|
|
|
+ }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ return task.result;
|
|
|
|
+}
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+BackupStatus *qmp_query_backup(Error **errp)
|
2020-03-10 17:12:50 +03:00
|
|
|
+{
|
|
|
|
+ BackupStatus *info = g_malloc0(sizeof(*info));
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_lock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+ if (!backup_state.stat.start_time) {
|
|
|
|
+ /* not started, return {} */
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
|
|
|
+ return info;
|
2020-03-10 17:12:50 +03:00
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ info->has_start_time = true;
|
|
|
|
+ info->start_time = backup_state.stat.start_time;
|
|
|
|
+
|
|
|
|
+ if (backup_state.stat.backup_file) {
|
|
|
|
+ info->backup_file = g_strdup(backup_state.stat.backup_file);
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ info->uuid = g_strdup(backup_state.stat.uuid_str);
|
|
|
|
+
|
|
|
|
+ if (backup_state.stat.end_time) {
|
|
|
|
+ if (backup_state.stat.error) {
|
|
|
|
+ info->status = g_strdup("error");
|
|
|
|
+ info->errmsg = g_strdup(error_get_pretty(backup_state.stat.error));
|
|
|
|
+ } else {
|
|
|
|
+ info->status = g_strdup("done");
|
|
|
|
+ }
|
|
|
|
+ info->has_end_time = true;
|
|
|
|
+ info->end_time = backup_state.stat.end_time;
|
|
|
|
+ } else {
|
|
|
|
+ info->status = g_strdup("active");
|
|
|
|
+ }
|
|
|
|
+
|
|
|
|
+ info->has_total = true;
|
|
|
|
+ info->total = backup_state.stat.total;
|
|
|
|
+ info->has_zero_bytes = true;
|
|
|
|
+ info->zero_bytes = backup_state.stat.zero_bytes;
|
|
|
|
+ info->has_transferred = true;
|
|
|
|
+ info->transferred = backup_state.stat.transferred;
|
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ qemu_mutex_unlock(&backup_state.stat.lock);
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
2021-03-03 12:56:02 +03:00
|
|
|
+ return info;
|
2020-03-10 17:12:50 +03:00
|
|
|
+}
|
|
|
|
diff --git a/qapi/block-core.json b/qapi/block-core.json
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
index 542add004b..16fb4c5ea0 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/qapi/block-core.json
|
|
|
|
+++ b/qapi/block-core.json
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -835,6 +835,115 @@
|
update submodule and patches to 7.1.0
Notable changes:
* The only big change is the switch to using a custom QIOChannel for
savevm-async, because the previously used QEMUFileOps was dropped.
Changes to the current implementation:
* Switch to vector based methods as required for an IO channel. For
short reads the passed-in IO vector is stuffed with zeroes at the
end, just to be sure.
* For reading: The documentation in include/io/channel.h states that
at least one byte should be read, so also error out when whe are
at the very end instead of returning 0.
* For reading: Fix off-by-one error when request goes beyond end.
The wrong code piece was:
if ((pos + size) > maxlen) {
size = maxlen - pos - 1;
}
Previously, the last byte would not be read. It's actually
possible to get a snapshot .raw file that has content all the way
up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
trailing zero bytes (I wrote a script to do it).
Luckily, it didn't cause a real issue, because qemu_loadvm_state()
is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
section. The buffer for reading it is simply freed up afterwards
and the function will assume that it read the whole section, even
if that's not the case.
* For writing: Make use of the generated blk_pwritev() wrapper
instead of manually wrapping the coroutine to simplify and save a
few lines.
* Adapt to changed interfaces for blk_{pread,pwrite}:
* a9262f551e ("block: Change blk_{pread,pwrite}() param order")
* 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
* bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
Those changes especially affected the qemu-img dd patches, because
the context also changed, but also some of our block drivers used
the functions.
* Drop qemu-common.h include: it got renamed after essentially
everything was moved to other headers. The only remaining user I
could find for things dropped from the header between 7.0 and 7.1
was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
already includes the header to which the function was moved.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
|
|
|
{ 'command': 'query-block', 'returns': ['BlockInfo'],
|
|
|
|
'allow-preconfig': true }
|
2020-03-10 17:12:50 +03:00
|
|
|
|
|
|
|
+##
|
|
|
|
+# @BackupStatus:
|
|
|
|
+#
|
|
|
|
+# Detailed backup status.
|
|
|
|
+#
|
|
|
|
+# @status: string describing the current backup status.
|
|
|
|
+# This can be 'active', 'done', 'error'. If this field is not
|
|
|
|
+# returned, no backup process has been initiated
|
|
|
|
+#
|
|
|
|
+# @errmsg: error message (only returned if status is 'error')
|
|
|
|
+#
|
|
|
|
+# @total: total amount of bytes involved in the backup process
|
|
|
|
+#
|
|
|
|
+# @transferred: amount of bytes already backed up.
|
|
|
|
+#
|
|
|
|
+# @zero-bytes: amount of 'zero' bytes detected.
|
|
|
|
+#
|
|
|
|
+# @start-time: time (epoch) when backup job started.
|
|
|
|
+#
|
|
|
|
+# @end-time: time (epoch) when backup job finished.
|
|
|
|
+#
|
|
|
|
+# @backup-file: backup file name
|
|
|
|
+#
|
|
|
|
+# @uuid: uuid for this backup job
|
|
|
|
+#
|
|
|
|
+##
|
|
|
|
+{ 'struct': 'BackupStatus',
|
|
|
|
+ 'data': {'*status': 'str', '*errmsg': 'str', '*total': 'int',
|
|
|
|
+ '*transferred': 'int', '*zero-bytes': 'int',
|
|
|
|
+ '*start-time': 'int', '*end-time': 'int',
|
|
|
|
+ '*backup-file': 'str', '*uuid': 'str' } }
|
|
|
|
+
|
|
|
|
+##
|
|
|
|
+# @BackupFormat:
|
|
|
|
+#
|
|
|
|
+# An enumeration of supported backup formats.
|
|
|
|
+#
|
|
|
|
+# @vma: Proxmox vma backup format
|
|
|
|
+##
|
|
|
|
+{ 'enum': 'BackupFormat',
|
|
|
|
+ 'data': [ 'vma', 'dir', 'pbs' ] }
|
|
|
|
+
|
|
|
|
+##
|
|
|
|
+# @backup:
|
|
|
|
+#
|
|
|
|
+# Starts a VM backup.
|
|
|
|
+#
|
|
|
|
+# @backup-file: the backup file name
|
|
|
|
+#
|
|
|
|
+# @format: format of the backup file
|
|
|
|
+#
|
|
|
|
+# @config-file: a configuration file to include into
|
2021-02-11 19:11:11 +03:00
|
|
|
+# the backup archive.
|
2020-03-10 17:12:50 +03:00
|
|
|
+#
|
|
|
|
+# @speed: the maximum speed, in bytes per second
|
|
|
|
+#
|
|
|
|
+# @devlist: list of block device names (separated by ',', ';'
|
2021-02-11 19:11:11 +03:00
|
|
|
+# or ':'). By default the backup includes all writable block devices.
|
2020-03-10 17:12:50 +03:00
|
|
|
+#
|
|
|
|
+# @password: backup server passsword (required for format 'pbs')
|
|
|
|
+#
|
|
|
|
+# @keyfile: keyfile used for encryption (optional for format 'pbs')
|
|
|
|
+#
|
|
|
|
+# @key-password: password for keyfile (optional for format 'pbs')
|
|
|
|
+#
|
|
|
|
+# @fingerprint: server cert fingerprint (optional for format 'pbs')
|
|
|
|
+#
|
|
|
|
+# @backup-id: backup ID (required for format 'pbs')
|
|
|
|
+#
|
|
|
|
+# @backup-time: backup timestamp (Unix epoch, required for format 'pbs')
|
|
|
|
+#
|
|
|
|
+# Returns: the uuid of the backup job
|
|
|
|
+#
|
|
|
|
+##
|
|
|
|
+{ 'command': 'backup', 'data': { 'backup-file': 'str',
|
|
|
|
+ '*password': 'str',
|
|
|
|
+ '*keyfile': 'str',
|
|
|
|
+ '*key-password': 'str',
|
|
|
|
+ '*fingerprint': 'str',
|
|
|
|
+ '*backup-id': 'str',
|
|
|
|
+ '*backup-time': 'int',
|
|
|
|
+ '*format': 'BackupFormat',
|
|
|
|
+ '*config-file': 'str',
|
|
|
|
+ '*firewall-file': 'str',
|
|
|
|
+ '*devlist': 'str', '*speed': 'int' },
|
|
|
|
+ 'returns': 'UuidInfo' }
|
|
|
|
+
|
|
|
|
+##
|
|
|
|
+# @query-backup:
|
|
|
|
+#
|
|
|
|
+# Returns information about current/last backup task.
|
|
|
|
+#
|
|
|
|
+# Returns: @BackupStatus
|
|
|
|
+#
|
|
|
|
+##
|
|
|
|
+{ 'command': 'query-backup', 'returns': 'BackupStatus' }
|
|
|
|
+
|
|
|
|
+##
|
|
|
|
+# @backup-cancel:
|
|
|
|
+#
|
|
|
|
+# Cancel the current executing backup process.
|
|
|
|
+#
|
|
|
|
+# Returns: nothing on success
|
|
|
|
+#
|
|
|
|
+# Notes: This command succeeds even if there is no backup process running.
|
|
|
|
+#
|
|
|
|
+##
|
|
|
|
+{ 'command': 'backup-cancel' }
|
|
|
|
+
|
|
|
|
##
|
|
|
|
# @BlockDeviceTimedStats:
|
|
|
|
#
|
|
|
|
diff --git a/qapi/common.json b/qapi/common.json
|
update submodule and patches to 7.1.0
Notable changes:
* The only big change is the switch to using a custom QIOChannel for
savevm-async, because the previously used QEMUFileOps was dropped.
Changes to the current implementation:
* Switch to vector based methods as required for an IO channel. For
short reads the passed-in IO vector is stuffed with zeroes at the
end, just to be sure.
* For reading: The documentation in include/io/channel.h states that
at least one byte should be read, so also error out when whe are
at the very end instead of returning 0.
* For reading: Fix off-by-one error when request goes beyond end.
The wrong code piece was:
if ((pos + size) > maxlen) {
size = maxlen - pos - 1;
}
Previously, the last byte would not be read. It's actually
possible to get a snapshot .raw file that has content all the way
up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
trailing zero bytes (I wrote a script to do it).
Luckily, it didn't cause a real issue, because qemu_loadvm_state()
is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
section. The buffer for reading it is simply freed up afterwards
and the function will assume that it read the whole section, even
if that's not the case.
* For writing: Make use of the generated blk_pwritev() wrapper
instead of manually wrapping the coroutine to simplify and save a
few lines.
* Adapt to changed interfaces for blk_{pread,pwrite}:
* a9262f551e ("block: Change blk_{pread,pwrite}() param order")
* 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
* bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
Those changes especially affected the qemu-img dd patches, because
the context also changed, but also some of our block drivers used
the functions.
* Drop qemu-common.h include: it got renamed after essentially
everything was moved to other headers. The only remaining user I
could find for things dropped from the header between 7.0 and 7.1
was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
already includes the header to which the function was moved.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
|
|
|
index 356db3f670..aae8a3b682 100644
|
2020-03-10 17:12:50 +03:00
|
|
|
--- a/qapi/common.json
|
|
|
|
+++ b/qapi/common.json
|
update submodule and patches to 7.1.0
Notable changes:
* The only big change is the switch to using a custom QIOChannel for
savevm-async, because the previously used QEMUFileOps was dropped.
Changes to the current implementation:
* Switch to vector based methods as required for an IO channel. For
short reads the passed-in IO vector is stuffed with zeroes at the
end, just to be sure.
* For reading: The documentation in include/io/channel.h states that
at least one byte should be read, so also error out when whe are
at the very end instead of returning 0.
* For reading: Fix off-by-one error when request goes beyond end.
The wrong code piece was:
if ((pos + size) > maxlen) {
size = maxlen - pos - 1;
}
Previously, the last byte would not be read. It's actually
possible to get a snapshot .raw file that has content all the way
up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
trailing zero bytes (I wrote a script to do it).
Luckily, it didn't cause a real issue, because qemu_loadvm_state()
is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
section. The buffer for reading it is simply freed up afterwards
and the function will assume that it read the whole section, even
if that's not the case.
* For writing: Make use of the generated blk_pwritev() wrapper
instead of manually wrapping the coroutine to simplify and save a
few lines.
* Adapt to changed interfaces for blk_{pread,pwrite}:
* a9262f551e ("block: Change blk_{pread,pwrite}() param order")
* 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
* bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
Those changes especially affected the qemu-img dd patches, because
the context also changed, but also some of our block drivers used
the functions.
* Drop qemu-common.h include: it got renamed after essentially
everything was moved to other headers. The only remaining user I
could find for things dropped from the header between 7.0 and 7.1
was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
already includes the header to which the function was moved.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2022-10-14 15:07:13 +03:00
|
|
|
@@ -206,3 +206,16 @@
|
2022-02-11 12:24:33 +03:00
|
|
|
##
|
|
|
|
{ 'struct': 'HumanReadableText',
|
|
|
|
'data': { 'human-readable-text': 'str' } }
|
2020-03-10 17:12:50 +03:00
|
|
|
+
|
|
|
|
+##
|
|
|
|
+# @UuidInfo:
|
|
|
|
+#
|
|
|
|
+# Guest UUID information (Universally Unique Identifier).
|
|
|
|
+#
|
|
|
|
+# @UUID: the UUID of the guest
|
|
|
|
+#
|
|
|
|
+# Since: 0.14.0
|
|
|
|
+#
|
|
|
|
+# Notes: If no UUID was specified for the guest, a null UUID is returned.
|
|
|
|
+##
|
|
|
|
+{ 'struct': 'UuidInfo', 'data': {'UUID': 'str'} }
|
2021-02-11 19:11:11 +03:00
|
|
|
diff --git a/qapi/machine.json b/qapi/machine.json
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
index 47f3facdb2..46760978ae 100644
|
2021-02-11 19:11:11 +03:00
|
|
|
--- a/qapi/machine.json
|
|
|
|
+++ b/qapi/machine.json
|
|
|
|
@@ -4,6 +4,8 @@
|
|
|
|
# This work is licensed under the terms of the GNU GPL, version 2 or later.
|
|
|
|
# See the COPYING file in the top-level directory.
|
|
|
|
|
|
|
|
+{ 'include': 'common.json' }
|
|
|
|
+
|
|
|
|
##
|
|
|
|
# = Machines
|
|
|
|
##
|
update submodule and patches to QEMU 8.0.0
Many changes were necessary this time around:
* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.
* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.
* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.
* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.
* Async snapshot-related changes:
- The pending querying got adapted to the above-mentioned split and
a patch is added to optimize it/make it more similar to what
upstream code does.
- Added initialization of the compression counters (for
future-proofing).
- It's necessary the hold the BQL (big QEMU lock = iothread mutex)
during the setup phase, because block layer functions are used there
and not doing so leads to racy, hard-to-debug crashes or hangs. It's
necessary to change some upstream code too for this, a version of
the patch "migration: for snapshots, hold the BQL during setup
callbacks" is intended to be upstreamed.
- Need to take the bdrv graph read lock before flushing.
* hmp_info_balloon was moved to a different file.
* Needed to include a new headers from time to time to still get the
correct functions.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-05-15 16:39:53 +03:00
|
|
|
@@ -228,19 +230,6 @@
|
2020-03-10 17:12:50 +03:00
|
|
|
##
|
2021-02-11 19:11:11 +03:00
|
|
|
{ 'command': 'query-target', 'returns': 'TargetInfo' }
|
2020-03-10 17:12:50 +03:00
|
|
|
|
|
|
|
-##
|
|
|
|
-# @UuidInfo:
|
|
|
|
-#
|
|
|
|
-# Guest UUID information (Universally Unique Identifier).
|
|
|
|
-#
|
|
|
|
-# @UUID: the UUID of the guest
|
|
|
|
-#
|
2021-05-27 13:43:32 +03:00
|
|
|
-# Since: 0.14
|
2020-03-10 17:12:50 +03:00
|
|
|
-#
|
|
|
|
-# Notes: If no UUID was specified for the guest, a null UUID is returned.
|
|
|
|
-##
|
|
|
|
-{ 'struct': 'UuidInfo', 'data': {'UUID': 'str'} }
|
|
|
|
-
|
|
|
|
##
|
|
|
|
# @query-uuid:
|
|
|
|
#
|