Skip to content

Podman volume backups

Keeping backups of your volume data in your containers is important and requires a bit of work, and if you are a home user like me or a small company you might not afford big expensive container backup solutions this script might be an option for you to use.

This script exports all your volumes with a built in command in podman for exporting volumes as .tar files and then you can either store them locally only for any given amount of days and / or send them to offsite storage over SFTP to keep a backup elsewhere.

Each volume will reside in it's own .tar file to make it easier to restore / import the volume data for a single container when needed.

Important to note is that depending on your OS, you might not be able to add the pysftp python module when running as root, here you will need to set up and virtual environment for your script and recommended is to create it under the same user as you are running your containers, i will explain below on how to do that.

Settings

There are some settings in the script that must be set to make it tick, some settings already has a default value set but can be changed as needed.

Backup folder to use during creation of volume exports and to store files locally.

vBckDir

Name prefix of files, _date and .tar is added at the end, ex. prefix_volumename_date.tar.

vFilePrefix

Keep local backup files after sent to SFTP server, if no than nothing is kept locally. (no/yes)

vKeepBackup

Number of days to keep local files before pruning the backup directory, relies on vKeepBackup.

vKeepDays

Should we send the files to a Sftp server, requires the pysftp python module. (no/yes)

vSendToSftp

User for the remote server.

vSftpUser

Password for the remote server.

vSftpPass

Destination folder on remote server.

vSftpDir

Remote server address.

vSftpHost

Remote server port.

vSftpPort

Run extra OS specific commands before backup. (no/yes)

vPreBckCmd

Run extra OS specific commands after backup. (no/yes)

vPostBckCmd

OS Commands

Depending on some settings above, these 2 must be populated as well, these are the OS commands you want to execute either before or after export of volumes and depends on the vPreBckCmd and / or vPostBckCmd being set to yes.

External OS commands to execute before continuing with the rest of the script.

vPreOsCmd

External OS commands to execute at the end of the script.

vPostOsCmd

These can contain multiple commands when needed, to run multiple command for example vPreOsCmd could look like this, there are no theoretical no limit to amount of commands you can add, but try to keep it to what is needed for the script.

vPreOsCmd=["command1","command2"]

Python virtual environment

A python virtual environment or venv as it is called is runtime version of python separated from the rest of the OS, this has several benefits, no extra modules are installed with extra permissions and you can lock it on a certain python version in case you need different versions on your system.

As of Ubuntu 23.04 this is a requirement, they have locked down the system python version and you cannot install any extra modules, might be other OS going this way so better to start using a venv anyway.

Let's assume that our user is podman and it is the account running our containers, like i used in my podman how-to, go and read it if you haven't.

Start with making sure you are placed in the home directory.

$ cd
$ pwd
/home/podman

Run the following command to create a venv for this script.

$ python3 -m venv .venv/volumebackup

And if your are going to send to SFTP do the following to get the pysftp module installed.

$ source .venv/volumebackup/bin/activate
$ pip3 install wheel
$ pip3 install pysftp
$ deactivate

Now we need to change the first line in the script from this.

#!/usr/bin/python3

To this, so we point it at our venv we just created.

#!/home/podman/.venv/volumebackup/bin/python

Remote ssh key...

You need to ssh to the remote server before running the script first time to accept the host key when using the SFTP option or the script will fail, and it must be with the user running the script later.

To run the script simply set the execution mode like below so that the owner (user & group) of the script has full permissions on the script and everybody else has none, recommended since there are passwords in the file.

$ chmod 770 ./volumebackup.py

And this enables you to run it like this.

$ ./filebackup.py

Now you should have everything set up for running volume backup, or almost, you still need the script, and here it comes...

The Script

The script has been tested against the following Podman & Python versions.

Podman: 3.4.4 / 4.3.1
Python: 3.4.4 / 3.11.4

Expand below to see full script and to copy it to your environment.

volumebackup.py - Click to see full script 
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
#!/usr/bin/python3

#################################################################
# Container volume backup script written in python.             #
#                                                               #
# Author: Marcus Uddenhed                                       #
# Revision: 1.0                                                 #
# Revision Date: 2023-09-25                                     #
# Requirements:                                                 #
# pysftp for SFTP functions, only if vSendToSftp is set to yes. #
#                                                               #
#################################################################

## Module imports.
from datetime import datetime
from time import time
import subprocess
import os

## Global variables.
vBckDir=""                      # Backup folder to use during creation of volume exports and to store files locally.
vFilePrefix=""                  # Name prefix of files, _date and .tar is added at the end, ex. prefix_volumename_date.tar.
vKeepBackup="no"                # Keep local backup files after sent to SFTP server, if no than nothing is kept locally.(no/yes)
vKeepDays="20"                  # Days to keep if you want to keep certain days locally, relies on vKeepBackup.
vSendToSftp="no"                # Should we send the files to a Sftp server.(no/yes)
vSftpUser=""                    # User for remote server.
vSftpPass=""                    # Password for remote server.
vSftpDir=""                     # Destination folder on remote server.
vSftpHost=""                    # Remote server adress.
vSftpPort="22"                  # Remote server port.
vPreBckCmd="no"                 # Run extra OS specific commands before backup.(no/yes)
vPostBckCmd="no"                # Run extra OS specific commands after backup.(no/yes)

# External OS commands to execute before continuing with the rest of the script.
vPreOsCmd=[""]

# External OS commands to execute at the end of the script.
vPostOsCmd=[""]

# Volume list command
vListCmd="podman volume list --format {{.Name}}"

# Volume export command
vExportCmd="podman volume export --output"

#### Script Action

## Import pysftp only if vSendToSftp set to yes.
if vSendToSftp == "yes":
    import pysftp

## Define global array for volume file names.
gNameArray = []

## Get current date
def funcDateString():
    # Returns the today string year, month, day.
    return datetime.now().strftime("%Y%m%d")

## Define function for Pre OS commands.
def funcExecutePreOsCmd(vPreOsCmd):
    try:
        if vPreBckCmd.casefold() == "yes":
            # iterate through each specified command.
            for vExecute in vPreOsCmd:
                subprocess.run(vExecute, shell=True, check=True)
            # Send info to console.
            print("OS commands has been executed...")
    except:
        # Send info to console.
        print("Could not execute OS command...")

## Define function for Pre OS commands.
def funcExecutePostOsCmd(vPostOsCmd):
    try:
        if vPostBckCmd.casefold() == "yes":
            # iterate through each specified command.
            for vExecute in vPostOsCmd:
                subprocess.run(vExecute, shell=True, check=True)
            # Send info to console.
            print("OS commands has been executed...")
    except:
        # Send info to console.
        print("Could not execute OS command...")

## Define function for exporting volumes.
def funcExportVolumes():
    try:
        # Mark gNameArray global
        global gNameArray
        # Get volume names.
        vGetList = subprocess.Popen(vListCmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        # Create an array with the names.
        vNameArray = vGetList.stdout.readlines()
        # iterate through each specified volume name.
        for vName in vNameArray:
            vSetTarFile=os.path.join(vBckDir, vFilePrefix + "_" + vName.decode("utf-8").strip() + "_" + funcDateString() + ".tar")
            # Fill global array.
            gNameArray.append(vSetTarFile)
            # Execute export af volumes.
            vCmd=(vExportCmd + " " + vSetTarFile + " " + vName.decode("utf-8").strip())
            subprocess.run(vCmd, shell=True, check=True)
            # Send info to console.
            print("Volume exported: ", vName.decode("utf-8").strip())
    except:
        # Send info to console
        print("Could not export one or more volumes...")

## Define function send to SFTP.
def funcSendToSftp(vHost, vPort, vUser, vPwd, vFolder):
    try:
        if vSendToSftp.casefold() == "yes":
            # Send info to console.
            print("Sending files to SFTP server...")
            # Convert port to integer, needed since we use "" above to keep it a bit more tidy.
            vIntPort=int(vPort)
            # Iterate through file name array and send files.
            for vFile in gNameArray:
                # Connect to SFTP server and upload file.
                with pysftp.Connection(host=vHost, port=vIntPort, username=vUser, password=vPwd) as sftp:
                    # Change directory.
                    with sftp.cd(vFolder):
                        # Upload file
                        sftp.put(vFile)
                # Send info to console.
                print("Uploaded: ", vFile)
            # Send info to console.
            print("Done sending files to SFTP server...")
    except:
        # Send info to console.
        print("Could not connect to server or upload file...")

## Define history function.
def funcKeepBackup(vGetDays, vGetDir):
    try:
        # Check if to keep a history or not.
        vIntDays=int(vGetDays)
        if vKeepBackup.casefold() == "yes":
            # Send info to console.
            print("Pruning backup folder, keeping", vIntDays, "days...")
            # Set today as current day.
            vTimeNow=time()
            # Remove files based on days to keep.
            for fname in os.listdir(vGetDir):
                if fname.startswith(vFilePrefix):
                    if os.path.getmtime(os.path.join(vGetDir, fname)) < vTimeNow - vIntDays * 86400:
                        os.remove(os.path.join(vGetDir, fname))
            # Send info to console.
            print("Done pruning backup folder...")
        elif vKeepBackup.casefold() == "no":
            # Send info to console.
            print("Removing all local backup files...")
            # Build file list and remove files.
            vSetFilePattern=os.path.join(vFilePrefix + "_")
            for fname in os.listdir(vGetDir):
                if fname.startswith(vSetFilePattern):
                    os.remove(os.path.join(vGetDir, fname))
            # Send info to console.
            print("Done removing all local backup files...")
    except:
        # Send info to console.
        print("Could not clean backup folder...")

## Call the pre OS command function and run only if vPreBckCmd is set to yes.
funcExecutePreOsCmd(vPreOsCmd)

## Call the volume export function.
funcExportVolumes()

## Call the Sftp function and upload files only if vSendToSftp is set to yes.
funcSendToSftp(vSftpHost, vSftpPort, vSftpUser, vSftpPass, vSftpDir)

## Call the post OS command function and run only if vPostBckCmd is set to yes.
funcExecutePostOsCmd(vPostOsCmd)

## Call the history function to enable automatic housekeeping in the backup folder.
funcKeepBackup(vKeepDays, vBckDir)

Changelog

ChangeLog - Expand to read 
Version     Date            Information
-------     ----            -----------
1.0         2023-09-25      Initial release.