pve-backup-watcher/README.md

pve-backup-watch

Wait for a Proxmox VE backup to finish, then restart the guest.

pve-backup-watch is a small, dependency-free Go CLI that runs on a Proxmox VE node. You give it one or more VMIDs. It watches the node's task list until the vzdump backup for each guest has finished, then restarts that guest. You don't need to know when the backup job will run or how long it will take.

Contents

• Features
• Requirements
• Install
• Usage
• Options
• How it works
• Exit codes
• Examples
• Things to know
• Versioning

Features

• Acts only on the VMIDs you name, never on the whole node
• Works with both QEMU VMs (qm) and LXC containers (pct)
• Restarts after any result by default, or only after a failed backup with -only-failed
• Ignores old results, so a failure from last week never triggers a restart
• Dry-run mode to preview exactly what would happen
• Configurable poll interval and timeout
• Stops cleanly on Ctrl-C or SIGTERM
• Meaningful exit codes for use in scripts and cron
• Single static binary, standard library only

Requirements

• A Proxmox VE node (run the tool on the node that hosts the guest)
• Root, or a user allowed to run pvesh, qm and pct
• pvesh, qm and pct on the PATH (standard on Proxmox VE)
• Go 1.21 or newer, only to build from source

Install

Build from source and copy the binary to the node:

make
scp pve-backup-watch root@your-node:/usr/local/bin/

make builds a static linux/amd64 binary and embeds the version, git commit and build date. If you build on the node itself, or want another architecture, run go build directly:

go build -ldflags "-X main.version=dev-0.0.1 -X 'main.commit=First Build' -X 'main.date=$(date +%Y-%m-%d\ %H:%M:%S)'" -o .pve-backup-watch .

A plain go build reports the version as dev.

Usage

pve-backup-watch [options] VMID [VMID...]

Options can appear before, between or after the VMIDs. With several VMIDs, they are handled one after another in the order given.

Options

Option	Default	Description
-n, -dry-run	off	Print the restart commands instead of running them. It still waits for the backup.
-t, -timeout	6h	Maximum time to wait per VMID. 0 waits forever. On timeout, that guest is not restarted.
-i, -interval	30s	How often to poll the task list.
-f, -only-failed	off	Restart only if the backup did not finish with status OK.
-w, -warn-ok	off	With -only-failed, treat WARNINGS: n as success.
-node	short hostname	Proxmox node name, if it differs from the hostname.
-v, -version		Print version, commit and build date, then exit.

Durations use Go syntax: 30s, 15m, 6h.

How it works

1. Reads the node's recent vzdump tasks with pvesh get /nodes/<node>/tasks --typefilter vzdump.
2. Picks the newest task whose id matches the VMID and that is either still running or finished after the tool was started.
3. Polls until that task has an end time, then reads its status (OK, WARNINGS: n, or an error message).
4. Decides whether to restart (always by default, or only on failure with -only-failed).
5. Restarts the guest:
   1. qm or pct unlock, to clear a leftover backup lock (errors are ignored)
   2. shutdown --timeout 120 --forceStop 1, a graceful shutdown with a forced stop after 120 seconds
   3. start

Temporary errors from pvesh or JSON parsing don't abort the wait. A warning is printed and the next poll tries again.

Exit codes

Code	Meaning
0	Every VMID was handled without error
1	At least one VMID failed: timeout, guest not found, or a failed shutdown or start
2	Invalid usage, such as no VMID given or a non-numeric VMID

Examples

Preview what would happen for VM 123:

pve-backup-watch -n 123

Restart VM 123 as soon as its backup finishes, whatever the result:

pve-backup-watch 123

Restart only if the backup failed, treating warnings as success:

pve-backup-watch -f -w 123

Watch several guests, checking every 15 seconds and giving up after 2 hours each:

pve-backup-watch -i 15s -t 2h 123 124 200

Run in the background and keep a log:

nohup pve-backup-watch 123 &> /var/log/pve-backup-watch-123.log &

Things to know

• Start it before or during the backup. It only reacts to a backup that is running or starts after the tool launches. If you start it after the backup has already finished, it waits for the next one (or times out).
• vzdump stop mode restarts the guest itself. In that mode, this tool causes a second restart. It is most useful with snapshot or suspend mode, or when you want a restart after the backup regardless.
• A restart will not fix a broken backup. If backups fail because of storage or network problems, using -only-failed on a schedule reboots the guest every time without solving the cause.
• Local node only. It looks at tasks on the node it runs on, and the guest must be on that node.
• Recent tasks only. It reads the latest 500 vzdump tasks on the node, which is plenty for a backup that is running now.
• Timeouts never restart. If the backup doesn't finish in time, the guest is left alone and the tool exits with code 1.

Versioning

This project follows Semantic Versioning. The current version is in the VERSION file and is embedded in the binary at build time. Check it with:

pve-backup-watch -version

The tool also prints its version at the start of every run, so log files show which version produced them. See CHANGELOG.md for the history.

To release a change, update VERSION, add an entry to CHANGELOG.md, and rebuild with make.