This is an old revision of the document!
Tony's Proxmox Reinstall and Migration Guide
Use case
There are many guides for doing “migrations” or “reinstalls” of proxmox online, sometimes with backups, sometimes with live copies etc.
My needs
I have an existing Proxmox 7 server running with a single boot SSD using LVM for PVE. I have additional SSDs for hosting my LXC and VM images, and hard drives that are bind-mounted into the LXCs where bulk storage is needed (eg: Immich and Nextcloud data directories).
While this has backups of the containers, (and the bulk storage is RAID Z1), I am a single SSD failure away from the system not booting, which I consider catastrophic when it is a 6 hour $1000 flight away (thank Air Canada for that one).
I would like to reinstall PVE on this system, using two SSDs with ZFS RAID 1 for PVE, as 1) ZFS fits in far more with my current setup handling snapshots seemlessly, and 2) it gives me redundant boot and EFI partitions should one of the SSD's outright fail.
What is different
- I have NO quorum, I use a single instance of PVE.
- I can NOT do a live migration, as I have a single host, and will be reusing the hardware.
- I do NOT want to install all my LXCs and VMs from a backup - they live on a separate SSD I will not be wiping, and I would like to reuse those pools as is.
- I do NOT want to backup and restore PVE, I am changing SSD configurations (1 drive LVM to 2 drive ZFS)
- My old setup had many levels of jank during a PVE 7 to 8 upgrade, avoid re-using unnecessary old configs
Again, what I want, is to reinstall PVE onto a new SSD with a new filesystem, reuse my existing VM files and pools already on the drives. No backup/restore of LXCs. Just a simple, take everything, plunk it into new PVE.
The solution
Turns out the solution is actually relatively simple.
Since I tried a few things first and re-did this process twice, let me put some thoughts:
- Proxmox reads the configs for your VMs and LXCs from /etc/pve/nodes.
- I setup a full Proxmox Backup Server as an LXC another PVE instance (different location - not clustered)
The easy way
- Make backups of everything you care about.
- IF YOU WILL BE RE-USING SSD'S, MAKE SURE YOU DD A BACKUP OF THEM!
- This one saved me and allowed me to repeat this procedure twice.
Traps
- Do NOT, try to blindly restore your /etc/pve
- Doing so will leave your /etc/pve in a bad state with keys that are lingering
- You will NOT be able to log in and will need to systemctl stop, clear /etc/pve and such
- Connection error 401: permission denied - invalid PVE ticket
- See my above point
- Do not try to RSYNC into /etc/pve
- It will fail. It is a FUSE mount and will refuse the temp files rsync makes.
- Use scp or an rsync flag to go direct to output file.
- Do NOT try to restore your PVE host from Proxmox Backup Server
- Restoring the backup will yield errors with file overwrites
- Allowing overwrites will break your configs in two and fail on /etc/pve
- NOTE: proxmox-backup-client doesn't even backup /etc/pve by default (it only does root) so it's no use if you forgot tha tanyway
- PROXMOX BACKUP SERVER IS A RED HERRING!!
- In fact, if you don't need to back up and restore VMs, and you've taken a DD of your old SSD, you don't need it.
- If it very useful for offsite backing up items, just not my current use case