See added patches for more info, overview:
0044: slightly increase PBS performance by reducing allocations
0045: slightly increase block-stream performance for Ceph
0046: don't crash with block-stream on RBD
0047: add alloc-track driver for live restore
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Saving dirty bitmaps from our savevm-async code didn't work, since we
use a coroutine which holds the iothread mutex already (upstream savevm
is sync, migration uses a thread). Release the mutex before calling the
one function that (according to it's documentation) requires the lock to
*not* be held: qemu_savevm_state_pending.
Additionally, loading dirty bitmaps requires a call to
dirty_bitmap_mig_before_vm_start after "loadvm", which the upstream
savevm does explicitly afterwards - do that too.
This is exposed via the query-proxmox-support property
"pbs-dirty-bitmap-savevm".
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
...instead of having them in the middle of the backup related patches.
These might (hopefully) become upstream at some point as well.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Use blk_unref to drop the last reference, which will close the block
backend and flush all caches and outstanding writes.
This is especially important for restoring to Ceph, as the userspace
librbd caches will not be flushed if the application exits immediately,
leading to potentially incomplete restores.
Reported-by: Eneko Lacunza <elacunza@binovo.es>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
it ships a symbol file now, so it can be auto-generated based on the
build-dep and usage.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Lots of patches touched and some slight changes to the build process
since QEMU switched to meson as their build system. Functionality-wise
very little rebasing required.
New patches introduced:
* pve/0058: to fix VMA backups and clean up some code in general with
new 5.2 features now available to us (namely coroutine-enabled QMP).
* extra/0002: don't build man pages for guest agent when disabled
* extra/0003: fix live-migration with hugepages
* 0017 and 0018 are adjusted to fix snapshot abort and improve
snap performance a bit
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Fixes vma restore when the target is an iSCSI storage which expects that
initiatorname. Also avoids the need to always explicitly set the initiatorname
in PVE code, thus fixing moving efidisks from and to such iSCSI storages.
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
jemalloc does not play nice with our Rust library (proxmox-backup-qemu),
specifically it never releases memory allocated from Rust to the OS.
This leads to a problem with larger caches (e.g. for the PBS block driver).
It appears to be related to this GitHub issue:
https://github.com/jemalloc/jemalloc/issues/1398
The background_thread solution seems weirdly hacky, so let's disable
jemalloc entirely for now.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
bump build-dependency on libproxmox-backup-qemu0-dev with version query
support
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
The idea in general is to migrate all the state, which is small for
us, in a single step once. But, QEMU only calls save state if we
return active true.
Hardcoding is-active to return true, like done initially, makes the
migration freeze, as QEMU thinks this is never done, and only stops
calling us and finishes after a few seconds.
So, add a state with an "active" boolean, set to true when
initializing a migration, and set it to false when the state was
saved.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
With the transaction patches, patch 0026-PVE-Backup-modify-job-api.patch
is no longer necessary, so drop it and rebase all following patches on
top.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Prints QEMU errors that occur *after* the "-daemonize" fork to the
systemd journal, instead of pushing them into /dev/null like before.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
this is needed for template backups with PBS until we have the backup
equivalent of 'pbs-restore'.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
No major semantic changes, mostly just deprecations and changed function
signatures. Drop the extra/ patches, as they have been applied upstream.
The added extra/ patch was accepted upstream[0] but has not been picked
up for 5.1. It is required for non-4M aligned backups to work with PBS.
[0] https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg01671.html
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
fixes an assertion failure in qemu network packet processing, which can
lead to DOS'ing the qemu process on the host. this affects 'e1000e' and
'vmxnet3' network devices.
patch is cherry-picked from the commit mentioned in the oss-security email.
more info on oss-security [0]
[0]: https://www.openwall.com/lists/oss-security/2020/08/10/1
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
"PVE backup: rename incremental to use-dirty-bitmap" merged two
variables (use_dirty_bitmap and incremental) into one, but they served
two different purposes. Rename the original use_dirty_bitmap to
"expect_only_dirty" so the new one doesn't conflict, and rework "PVE:
use proxmox_backup_check_incremental" around that semantic.
In practice, this had the effect that only one disk at a time would
have a bitmap added, as after the first "use_dirty_bitmap" would be set
to one and the rest would behave as if the QMP parameter of the same
name was unset.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
