Commit Graph

268 Commits

Author SHA1 Message Date
Stefan Reiter
817b7667e8 Update to QEMU 5.2
Lots of patches touched and some slight changes to the build process
since QEMU switched to meson as their build system. Functionality-wise
very little rebasing required.

New patches introduced:
* pve/0058: to fix VMA backups and clean up some code in general with
  new 5.2 features now available to us (namely coroutine-enabled QMP).
* extra/0002: don't build man pages for guest agent when disabled
* extra/0003: fix live-migration with hugepages
* 0017 and 0018 are adjusted to fix snapshot abort and improve
  snap performance a bit

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-12 10:20:01 +01:00
Fabian Ebner
a16eaaffd3 fix #3084: fall back to open-iscsi initiatorname
Fixes vma restore when the target is an iSCSI storage which expects that
initiatorname. Also avoids the need to always explicitly set the initiatorname
in PVE code, thus fixing moving efidisks from and to such iSCSI storages.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-02-06 15:09:15 +01:00
Wolfgang Bumiller
cfa22a5b2a bump version to 5.1.0-8
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-07 10:28:46 +01:00
Wolfgang Bumiller
b515d45e6b fix #3225: properly cancel jobs in 'created' state
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-07 10:26:37 +01:00
Stefan Reiter
3d785eabbe disable jemalloc
jemalloc does not play nice with our Rust library (proxmox-backup-qemu),
specifically it never releases memory allocated from Rust to the OS.
This leads to a problem with larger caches (e.g. for the PBS block driver).

It appears to be related to this GitHub issue:
https://github.com/jemalloc/jemalloc/issues/1398

The background_thread solution seems weirdly hacky, so let's disable
jemalloc entirely for now.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-12-15 14:43:34 +01:00
Thomas Lamprecht
55df4a9eb1 bump version to 5.1.0-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-25 14:09:24 +01:00
Stefan Reiter
cfe02b3b4e update patches with some pbs-state migration cleanups
...and literal cleanup, as in, call save_cleanup after success or error.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-25 11:49:06 +01:00
Stefan Reiter
32ee41155b update patches with squashed in 'include library version'
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>

bump build-dependency on libproxmox-backup-qemu0-dev with version query
support

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-25 11:49:06 +01:00
Thomas Lamprecht
424f6841d5 bump version to 5.1.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-05 18:59:44 +01:00
Thomas Lamprecht
66eae0ae75 fix dirty-bitmap state migration freeze
The idea in general is to migrate all the state, which is small for
us, in a single step once. But, QEMU only calls save state if we
return active true.

Hardcoding is-active to return true, like done initially, makes the
migration freeze, as QEMU thinks this is never done, and only stops
calling us and finishes after a few seconds.

So, add a state with an "active" boolean, set to true when
initializing a migration, and set it to false when the state was
saved.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-05 18:58:15 +01:00
Thomas Lamprecht
2130e925a8 bump version to 5.1.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-04 18:36:42 +01:00
Thomas Lamprecht
f36fa39113 migration/block-dirty-bitmap: migrate other bitmaps even if one fails
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-04 18:35:50 +01:00
Thomas Lamprecht
50eb3ea618 bump version to 5.1.0-4
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-29 18:09:22 +01:00
Thomas Lamprecht
3e2a0699e6 d/control: update versioned dependency for proxmox backup qemu library
to have the proxmox_export_state et al. available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-29 18:07:14 +01:00
Thomas Lamprecht
d95ad93eed apply dirty-bitmap state migration + fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-29 18:05:43 +01:00
Stefan Reiter
72ae34ecce Several fixes for backup abort and error reporting
Also add my Signed-off-by to some patches where it was missing.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-29 17:57:47 +01:00
Thomas Lamprecht
60a607ad0f bump version to 5.1.0-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-29 09:23:06 +02:00
Stefan Reiter
d333327a1b Add transaction patches and fix for blocking finish
With the transaction patches, patch 0026-PVE-Backup-modify-job-api.patch
is no longer necessary, so drop it and rebase all following patches on
top.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-29 09:21:15 +02:00
Thomas Lamprecht
250d87c276 bump version to 5.1.0-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-14 20:01:32 +02:00
Thomas Lamprecht
4b7a18845c cherry-pick: "usb: fix setup_len init (CVE-2020-14364)"
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-14 19:38:34 +02:00
Thomas Lamprecht
7895b0d523 work around #3002: revert "qemu-img convert: Don't pre-zero images"
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-14 19:37:45 +02:00
Stefan Reiter
437d68473c Add systemd journal logging patch
Prints QEMU errors that occur *after* the "-daemonize" fork to the
systemd journal, instead of pushing them into /dev/null like before.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-08 17:13:29 +02:00
Thomas Lamprecht
46c0ec19ad bump version to 5.1.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-20 14:48:58 +02:00
Fabian Grünbichler
a5feeebc51 allow backup of read-only block drives
this is needed for template backups with PBS until we have the backup
equivalent of 'pbs-restore'.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-20 14:11:39 +02:00
Stefan Reiter
60ae3775bf update to QEMU 5.1
No major semantic changes, mostly just deprecations and changed function
signatures. Drop the extra/ patches, as they have been applied upstream.

The added extra/ patch was accepted upstream[0] but has not been picked
up for 5.1. It is required for non-4M aligned backups to work with PBS.

[0] https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg01671.html

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-20 13:40:36 +02:00
Thomas Lamprecht
878df11e78 bump version to 5.0.0-13
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 18:19:35 +02:00
Thomas Lamprecht
f00a720d7e PVE: add query-pbs-bitmap-info QMP call
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 18:11:23 +02:00
Thomas Lamprecht
c5f7dc1d72 PVE: add zero block handling to PBS dump callback
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 13:56:03 +02:00
Fabian Grünbichler
dde77467ca bump version to 5.0.0-12
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 11:29:21 +02:00
Fabian Grünbichler
2821f02d70 fix PBS write callback with big blocks
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 11:14:36 +02:00
Oguz Bektas
95fd47ecb9 patch for possible DOS in qemu network packet processing
fixes an assertion failure in qemu network packet processing, which can
lead to DOS'ing the qemu process on the host. this affects 'e1000e' and
'vmxnet3' network devices.

patch is cherry-picked from the commit mentioned in the oss-security email.

more info on oss-security [0]

[0]: https://www.openwall.com/lists/oss-security/2020/08/10/1

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-08-11 11:08:39 +02:00
Thomas Lamprecht
41424ed9c8 bump version to 5.0.0-11
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-14 12:45:24 +02:00
Stefan Reiter
f257cc05f4 Fix dirty-bitmap PBS backup with multiple drives
"PVE backup: rename incremental to use-dirty-bitmap" merged two
variables (use_dirty_bitmap and incremental) into one, but they served
two different purposes. Rename the original use_dirty_bitmap to
"expect_only_dirty" so the new one doesn't conflict, and rework "PVE:
use proxmox_backup_check_incremental" around that semantic.

In practice, this had the effect that only one disk at a time would
have a bitmap added, as after the first "use_dirty_bitmap" would be set
to one and the rest would behave as if the QMP parameter of the same
name was unset.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-14 10:46:48 +02:00
Wolfgang Bumiller
9886892f10 bump version to 5.0.0-10
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 13:33:44 +02:00
Wolfgang Bumiller
6d46b2ff4c fix backup qmp parameters to pass along encryption info
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 13:31:52 +02:00
Thomas Lamprecht
2e4f5f2a90 bump version to 5.0.0-9
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 13:18:48 +02:00
Thomas Lamprecht
1d606ec161 d/control: update build-dependency of libproxmox-backup-qemu0-dev
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 13:17:29 +02:00
Thomas Lamprecht
3499c5b45a PBS patches: block driver, adapat encrypt/compress param, add query-proxmox-support QMP cmd
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-09 13:15:49 +02:00
Thomas Lamprecht
102ddd7e59 bump version to 5.0.0-8
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 23:05:42 +02:00
Thomas Lamprecht
4c17eebee4 fixup: proxmox_backup_check_incremental is negated
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 23:05:42 +02:00
Thomas Lamprecht
dfed71b229 bump version to 5.0.0-7
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 22:19:12 +02:00
Thomas Lamprecht
3ab149ccdd update/add PBS integration patches
* rename "incremental" param to "use-dirty-bitmap", avoids confusion
  as the backup can be incrementally also with that param set to
  false.
* use new proxmox_backup_check_incremental
* fix setting dirty counter and adapt to new connect API semantic

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-06 22:13:12 +02:00
Thomas Lamprecht
1f8140323f bump version to 5.0.0-6
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-03 19:26:09 +02:00
Thomas Lamprecht
15b9c76e1f pbs: query-backup: set reused field also for dirty-bitmap
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-03 19:26:09 +02:00
Thomas Lamprecht
d7f4e01a34 debian/patches: squash some followup patches and regroup a bit more together
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-02 13:33:16 +02:00
Thomas Lamprecht
8efe995b49 fixup: qemu submodule should be at v5.0.0
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-02 13:33:16 +02:00
Thomas Lamprecht
20be7fa0a0 backup: improve QAPI info and remove all dirty-bitmaps on failed drive-job
effectively two commits merged as one:
https://pve.proxmox.com/pipermail/pve-devel/2020-July/044185.html
https://pve.proxmox.com/pipermail/pve-devel/2020-July/044194.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-02 13:03:49 +02:00
Thomas Lamprecht
0943af81a6 bump version to 5.0.0-5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-30 12:59:26 +02:00
Thomas Lamprecht
04e436ea6b d/control: build depend on newer libproxmox-backup-qemu0-dev
to ensure we have the new 'incremental' parameter for the
proxmox_backup_register_image and proxmox_backup_register_image_async
functions available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-30 12:53:01 +02:00
Dietmar Maurer
c96a4a38cb add incremental backup patches
and fix typo: s/BPS/PBS/
2020-06-30 10:34:00 +02:00