Podman pod & container migration script

Version: 1.3.2 Date: 2024-04-12

In my endeavor of finding a good tool for migration containers and pods with podman rootless i decided to build my own solution based on python since i did not find anything that had a minimum impact on my podman environment.

This script do not require much of additional tools or configuration to work.

This script will not work in every situation, but i have tried to make it as good as possible, but i'm always open for ideas and input for further development of this script to take it even further.

The script is built around standard podman commands and should be fairly version compatible.

When migrating a pod with this script it will migrate all of the containers belonging to that pod except the one called infra, that container is recreated on the remote server again during migration and is non transferable.

Spotting issues...

If you find any issues with this script, do not hesitate to get in contact, i will look into the matter with upmost urgency so that this script remains usable for you.

Check out the github page for this script to see if the issue has already been spotted or to file a new issue report.

Settings

There are some settings in the script that must be set to make it tick, some settings already has a default value but can be changed if needed.

vMigrateDir

Where to put the files during migration for further processing, path must exist on both servers and consist of an absolute path, using ~/.. is not recommended and can yield weird results. cleaning this directory between migrations is recommended, even if the migration failed.

No other files may exist in this folder, since this can yield unexpected results and if setting vCleanMigrateDir to yes will erase all files within this directory.

vCleanMigrateDir

Default this is set to No and you manually need to clean the migration folder, but if you want to clean out the folder after each migration you can set this to yes and the script will do it for you.

vEnvDir

Where container env parameter files resides on both servers, must match on both sides and is not a temporary folder, the names of each env file put here for migration must match what is set on the container or it will fail.

vSecDir

Secret directory, where secret files reside during migration only, each file must match the secret name(s) used on the container. If you manually want to create the secret on the remote server you can do so and set this to empty, just remember that the secret name created on the remote server must match what is set on the container or it will fail.

vFilePrefix

Prefix for image & volume backups during migration, do not add a _ at the end, that is added later when building the file names.

vSftpUseKeyFile

Use a keyfile or username / password when connecting to a remote server, when set to No the script will ask for username and password and when set to Yes it requirers that vSftpKeyFilePath is set and the use of --keyfile parameter.

vSftpKeyFilePath

Set to full path where your key files reside, only used if vSftpUseKeyFile is set to Yes.

vAcceptDisclaimer

Set to Yes to not show the disclaimer, but please read it first.

Python virtual environment

A python virtual environment or venv as it is called is runtime version of python separated from the rest of the OS. This has several benefits like no extra modules are installed with extra permissions and you can lock it on a certain python version in case you need different versions on your system.

As of Ubuntu 23.04 this is a requirement, they have locked down the system python version and you cannot install any extra modules, might be other distributions going this way or is already there, it is better to start using a venv anyway.

Before creating the virtual environment you might need to install pythons venv package, the best thing is to try to create the venv and if it fails it will most of the times tell you what package it wants installed, install that package and try again.

Let's assume that our user is podman and it is the account running our containers, like i used in my podman how-to, go and read it if you haven't.

Start with making sure you are placed in the home directory.

$ cd
$ pwd
/home/podman

Run the following command to create a venv for this script, i set the name to oMigrate since it is an abbreviation on my site name and script function, but you can call it whatever you like.

$ python3 -m venv .venv/oMigrate

If above commands fails after you installed required package remove the oMigrate folder from .venv/ directory and try again.

We need the paramiko ssh module to be able to both transfer files between the servers and to run various commands on remote server during migration.

$ source .venv/oMigrate/bin/activate
$ pip3 install paramiko
$ deactivate

Now we need to change the first line in the script from this.

#!/usr/bin/python3

To this, so we point it at our venv we just created.

#!/home/podman/.venv/oMigrate/bin/python

Permissions

To run the script simply set the execution mode like below so that the owner (user & group) of the script has full permissions on the script and everybody else has none.

$ chmod 770 ./oMigrate.py

Running the script

To do a migration of an pod or container the script needs some required input parameters.

--type

Here you set if it is a pod or container your migrating.

--name

The name of the pod or container that is going to be migrated.

--dst

The destination server that shall receive the pod or container.

--port

The SSH port for the remote server.

--keyfile

The name of the keyfile to use.

All but one of these are always required to be able to run the script, the --keyfile is only required when the script is set to use a keyfile instead of username / password when connecting.

Heads Up...

Containers where you have manually modified any files within the containers own filesystem and where those files do not reside on a mounted volume that can be exported will be lost running a migration.

Examples

Migration of a pod and not using a keyfile.

./oMigrate.py --type "pod" --name "ExPod" --dst "10.10.10.10" -port "22"

Migration of a single container using the --keyfile option.

./oMigrate.py --type "container" --name "ExContainer" \
--dst "10.10.10.10" -port "22" --keyfile "KeyfileName"

The \ in above example is used only if you need to split your command over several lines in bash.

The Script

The script has been tested against the following Podman & Python versions.

Podman: 4.3.1
Python: 3.11.6

GitHub

This script now resides on GitHub and newer versions will be uploaded there from now on.

This page will still exist with information about the script and how to configure it for now...

You can reach it here: GitHub

Changelog

Here you can see the changes made to this script.

ChangeLog - Expand to read

Version   Date        Information
-------   ----------  -----------
1.3.2     2024-04-12  Added option to auto clean migration folders if migration finished.
                      Minor bug fixes.
1.3.1     2024-04-08  Refactored most of the code and added type annotations.
1.3.0     2024-04-07  Now handles dependencies of --requires against other containers
                      when used in a pod.
                      Fixed check for if a container is member of a pod.
                      Minor code corrections.
1.2.2     2024-04-06  Corrected output step numbering being slightly of.
                      Fixed double exit error info output for container pod membership check.
                      Fixed minor spelling errors.
1.2.1     2024-03-31  Added control for migrate folder being empty or not.
                      Improved container image sync, skip if already exist.
                      Fixed minor spelling errors.
1.2.0     2024-03-31  Added handling of secrets into the script.
                      Added handling of networks into the script.
                      Cleaned output to console to make each step clearer.
                      Refactored some code to make it work better.
                      Fixed som minor output bugs.
                      Added parsing of short commands like -v and so on.
1.1.0     2024-03-24  Added volume backup on pods,
                      Cleaned up info & error output,
                      Corrected version information.
1.0.0     2024-03-21  Initial release.