Commit Graph

74 Commits

Author SHA1 Message Date
Stefan Reiter
72ae34ecce Several fixes for backup abort and error reporting
Also add my Signed-off-by to some patches where it was missing.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-29 17:57:47 +01:00
Stefan Reiter
d333327a1b Add transaction patches and fix for blocking finish
With the transaction patches, patch 0026-PVE-Backup-modify-job-api.patch
is no longer necessary, so drop it and rebase all following patches on
top.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-29 09:21:15 +02:00
Stefan Reiter
437d68473c Add systemd journal logging patch
Prints QEMU errors that occur *after* the "-daemonize" fork to the
systemd journal, instead of pushing them into /dev/null like before.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-08 17:13:29 +02:00
Fabian Grünbichler
a5feeebc51 allow backup of read-only block drives
this is needed for template backups with PBS until we have the backup
equivalent of 'pbs-restore'.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-20 14:11:39 +02:00
Stefan Reiter
60ae3775bf update to QEMU 5.1
No major semantic changes, mostly just deprecations and changed function
signatures. Drop the extra/ patches, as they have been applied upstream.

The added extra/ patch was accepted upstream[0] but has not been picked
up for 5.1. It is required for non-4M aligned backups to work with PBS.

[0] https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg01671.html

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-20 13:40:36 +02:00
Thomas Lamprecht
f00a720d7e PVE: add query-pbs-bitmap-info QMP call
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 18:11:23 +02:00
Thomas Lamprecht
c5f7dc1d72 PVE: add zero block handling to PBS dump callback
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 13:56:03 +02:00
Fabian Grünbichler
2821f02d70 fix PBS write callback with big blocks
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 11:14:36 +02:00
Stefan Reiter
f257cc05f4 Fix dirty-bitmap PBS backup with multiple drives
"PVE backup: rename incremental to use-dirty-bitmap" merged two
variables (use_dirty_bitmap and incremental) into one, but they served
two different purposes. Rename the original use_dirty_bitmap to
"expect_only_dirty" so the new one doesn't conflict, and rework "PVE:
use proxmox_backup_check_incremental" around that semantic.

In practice, this had the effect that only one disk at a time would
have a bitmap added, as after the first "use_dirty_bitmap" would be set
to one and the rest would behave as if the QMP parameter of the same
name was unset.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-14 10:46:48 +02:00
Wolfgang Bumiller
6d46b2ff4c fix backup qmp parameters to pass along encryption info
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 13:31:52 +02:00
Thomas Lamprecht
3499c5b45a PBS patches: block driver, adapat encrypt/compress param, add query-proxmox-support QMP cmd
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 13:15:49 +02:00
Thomas Lamprecht
4c17eebee4 fixup: proxmox_backup_check_incremental is negated
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 23:05:42 +02:00
Thomas Lamprecht
3ab149ccdd update/add PBS integration patches
* rename "incremental" param to "use-dirty-bitmap", avoids confusion
  as the backup can be incrementally also with that param set to
  false.
* use new proxmox_backup_check_incremental
* fix setting dirty counter and adapt to new connect API semantic

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 22:13:12 +02:00
Thomas Lamprecht
15b9c76e1f pbs: query-backup: set reused field also for dirty-bitmap
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-03 19:26:09 +02:00
Thomas Lamprecht
d7f4e01a34 debian/patches: squash some followup patches and regroup a bit more together
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-02 13:33:16 +02:00
Thomas Lamprecht
20be7fa0a0 backup: improve QAPI info and remove all dirty-bitmaps on failed drive-job
effectively two commits merged as one:
https://pve.proxmox.com/pipermail/pve-devel/2020-July/044185.html
https://pve.proxmox.com/pipermail/pve-devel/2020-July/044194.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-02 13:03:49 +02:00
Dietmar Maurer
c96a4a38cb add incremental backup patches
and fix typo: s/BPS/PBS/
2020-06-30 10:34:00 +02:00
Thomas Lamprecht
b570f1c41e Fix backup for not 64k-aligned storages
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Roland Kammerer <roland.kammerer@linbit.com>
2020-06-24 16:26:30 +02:00
Thomas Lamprecht
fff7e250ee pbs-restore: flush verbose log before calling into library
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-02 13:20:56 +02:00
Thomas Lamprecht
bce72611f9 pbs-restore: be more verbose if asked to
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 18:48:00 +02:00
Thomas Lamprecht
c6979241f1 small cleanups for pbs-restore
Add trailing newline to two error messages, and drop an extra
unconditional `qdict_put_str(options, "driver", format);`
Besides that it's just formatting.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 15:24:20 +02:00
Thomas Lamprecht
f063a8aadb fix vmstate-snapshots with iothread=1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-27 18:54:06 +02:00
Wolfgang Bumiller
abc9e57f05 add optional buffer size to QEMUFile
and use 4M for our savevm-async buffer size

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-05-04 15:09:10 +02:00
Wolfgang Bumiller
24e289d025 experimentally move savevm-async back into a coroutine
Move qemu_savevm_state_{header,setup} into the main loop and
the rest of the iteration into a coroutine. The former need
to lock the iothread (and we can't unlock it in the
coroutine), and the latter can't deal with being in a
separate thread, so a coroutine it must be.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-05-04 15:09:10 +02:00
Thomas Lamprecht
751ed3661b avoid using QemuRecMutex inside coroutines, QemuMutex to lock outside
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-17 09:09:12 +02:00
Thomas Lamprecht
b7e851a529 rebase patches onto v5.0.0-rc3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-16 15:19:31 +02:00
Thomas Lamprecht
83faa3fe30 import QEMU 5.0.0-rc2 and rebase patches
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-04-08 15:36:32 +02:00
Fabian Grünbichler
059a9447e2 add bitmap drive-mirror patches
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-23 10:15:22 +01:00
Dominic Jäger
66e3df5d45 vma_writer: Display more error information
Also print the reason why the function vma_writer_register_stream failed to
help debug errors like in [0].

[0] https://forum.proxmox.com/threads/backup-error-vma_writer_register_stream-drive-scsi0-failed-pve-6-1-7.65925/

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
2020-03-19 18:58:47 +01:00
Dietmar Maurer
20505f521e fixup backup with io-threads 2020-03-12 10:30:18 +01:00
Thomas Lamprecht
6402d96100 update submodule and patches for 4.2.0
The long overdue nice rebase+cleanup was done by Dietmar

Originally-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-11 07:26:21 +01:00
Dietmar Maurer
84403c2d53 improve qemu backup by reducing lock contention
- reducing lock contention by using CoRwLock
- correctly call aio_wait_kick()
2020-02-18 10:47:21 +01:00
Thomas Lamprecht
2c67b15290 followup: fix indentation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-01-25 16:17:23 +01:00
Tim Marx
f47a9db42a fix hmp info backup command
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2020-01-25 16:09:47 +01:00
Thomas Lamprecht
9714cead2c grammar fix: s/does not exists/does not exist/g
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-13 12:10:51 +01:00
Thomas Lamprecht
a0866d18a5 fixup patch author
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-25 11:49:06 +01:00
Thomas Lamprecht
ac2969b218 allow to pass PVE version to machine
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-25 09:12:48 +01:00
Thomas Lamprecht
99b86f4f9d handle virtio-balloon cfg size compat with our 4.0 machines
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-22 10:07:39 +01:00
Thomas Lamprecht
917265984c update diffs/indexes from patches
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-22 10:07:39 +01:00
Stefan Reiter
1dc952cf51 Update and rebase to QEMU 4.1.1
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-11-21 07:59:09 +01:00
Stefan Reiter
a9b8206ded Apply fixups for 4.1
* Fix VMA tool build
* Change PVE code to new blockjob API
* Acquire missing lock for block_job_add_bdrv

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-11-21 07:59:05 +01:00
Stefan Reiter
be901f6656 Update and rebase to QEMU 4.1
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-11-21 07:58:57 +01:00
Wolfgang Bumiller
d4824fa37a fix backup speed regression
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-04 14:23:17 +01:00
Thomas Lamprecht
22ff24871d various small pve backup co-routine related fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-30 16:03:56 +01:00
Thomas Lamprecht
a5db0e22d7 backup: add various cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-24 08:49:09 +02:00
Thomas Lamprecht
cbb547903c run backup related code inside co-routines and improve locking
Patches-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-23 09:14:51 +02:00
Thomas Lamprecht
b8d43c594b update sources for v4.0.1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-22 09:52:03 +02:00
Dietmar Maurer
69cb18950a fix #1071: use correct AioContext to cope with IOThreads
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2019-10-21 15:51:43 +02:00
Wolfgang Bumiller
23ea029343 disable oob again
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-10-10 11:03:20 +02:00
Wolfgang Bumiller
3b1986f02f merge monitor oob fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-10-02 10:16:40 +02:00