Containers made it very easy to package and run applications on different Linux based platforms. Building and running a container can be done in seconds and is easy to reproduce. But what about updating the running containers on a regular basis?
This article explains how you can run containers with Podman and also keep them updated without any manual step.
Why (not) Auto Update
Updating a Linux based system can be done with a few commands or even automatically. For example, if you want to update a CentOS based system, you can execute the below commands, and you are good to go.
# Update and reboot $ sudo dnf update -y $ sudo reboot
(Un)fortunately, running containers will not be updated with the above commands. Instead, you have to update each an every container separately. For example, updating a WordPress setup can look like this.
# Update and restart a WordPress container setup $ podman image pull docker.io/library/wordpress:TAG $ podman image pull docker.ui/library/mariadb:TAG $ podman container rm -f myWordpressApp $ podman container rm -f myWordpressDb $ podman container run MANYOPTIONS docker.ui/library/mariadb:TAG $ podman container run MANYOPTIONS docker.ui/library/wordpress:TAG
Docker Compose makes it a bit easier, but even then, you have to do it for each and every stack. For a production environment, where you really want to use specific image versions and full control or use some automation (ex. Ansible), this seems ok. For an environment, where you tested every setup anyway, or you don't care if something may be broken, and you are able to recover on your own, it's just tedious work.
Podman provides a very easy way to keep your container setups updated automatically or On-demand. Let's see how this works.
Podman and Systemd
The first thing, you need to know about the auto-update capabilities: It only works with systemd managed containers. I have already written an article about this here, so I will stick to the additions to make auto-updates working. Here is a fast-forward example to generate a (rootfull) container (httpd), which is managed via systemd.
# Run container and publish port 80 $ sudo podman container run --detach --tty --name web01 --publish 80:80/tcp docker.io/library/httpd:latest # Test $ curl localhost:80 <html><body><h1>It works!</h1></body></html> # Generate systemd file and enable it $ sudo podman generate systemd --name --new --files web01 /home/dschier/container-web01.service $ sudo podman container rm -f web01 $ sudo cp /home/dschier/container-web01.service /etc/systemd/system/ $ sudo systemctl daemon-reload $ sudo systemctl enable --now container-web01 # Test again $ curl localhost:80 <html><body><h1>It works!</h1></body></html>
The generated file container-web01.service will look something like the below example. I just added some newlines to make it better readable.
Now, that we are having a systemd managed container, we do have 3 easy options to update the containers.
You can update a container with only two commands.
# Update the image and restart a container $ sudo podman pull docker.io/library/httpd:latest $ sudo systemctl restart container-web01
This is very useful, if you want to keep control over the update process itself and update only a single container. Be aware, that all containers, using this image will be updated with the next restart, too.
Auto-Update on demand
A more automated way can be achieved, if you tweak the systemd unit file very slightly. In fact, you only have to add a label to the container. The below example shows the added line
--label "io.containers.autoupdate=registry". This will ensure Podman is checking the fully qualified image path for new images and download them.
After editing your unit file, you need to make systemd aware of the change and also restart the container service.
# Reload systemd unit files and restart the container $ sudo systemctl daemon-reload $ sudo systemctl restart container-web01.service
Now comes the fun part. Podman provides a new sub-command
auto-update. This command will look for containers that are having the label and a systemd service file as described above. If the command finds one container, it will check for a new image, download it, restart the container service. So, just run the below command like you are running
$ sudo podman auto-update
Be aware, that this only updates the services for the user executing it. If you are running it with privileges (root/sudo), it will update only rootfull containers. Same, for a user executing the command.
Auto-Update on a schedule
There is even more to it. In smaller environments, you may want to "don't care" about updating your containers. You can enable a systemd timer, which is taking care of running
podman auto-update on a schedule. Let me show you, how to run automatic container updates every Monday morning for all rootfull containers.
First, you need to manage the container with systemd, as described above. In addition, you need to enable the podman-auto-update.timer. This can be done with the below command:
# Enable the auto update timer $ sudo systemctl enable --now podman-auto-update.timer
You can have a short look at the timer via
systemctl cat podman-auto-update.timer.
As you can see, the timer will run every day. I want to adjust this to every Monday morning. Changing system default unit files is really easy and can be done with the below command.
# Edit the timer file $ sudo systemctl edit podman-auto-update.timer
This will open your default editor and provide the original file as comments at the end. You just need to edit your changes, not provide a new file. In my case, I will edit the file with the below content.
[Timer] OnCalendar=Mon *-*-* 05:00 Persistent=true
This will ensure, that the auto-update is running every Monday at ~05:00 in the morning. You can use other expressions, depending on your needs, as described in the man pages.
# Look at the manpage for systemd timers $ man systemd.timer
Just blindly automating image updates and restarting containers may lead to unexpected behavior and issues. Here are some best practices, I want to provide to you. This is just my personal experience, so don't take them as industry standard.
- only use the label for uncritical workloads, container images can be broken, too
- don't use images with the latest tag, instead choose a versioned tag like 1, 1.1 or similar, if possible
- check, if you can really restart every image automatically or if these depend on each other and may crash, if something gets lost
- do backups of your volumes on a regular basis
- introduce some kind of monitoring, that informs you, if a container does not come up automatically
- be aware, that all containers using a specific image/tag will be updated with the next restart of the container
Furthermore, using automatic updates in a critical production environment should be avoided entirely. In most cases, you want to know when something is deployed and have full control over the deployment process.
Docs & Links
You can find a couple of articles, documentation and even videos about systemd timers or Podman auto-updates in the web.
With Podman auto-updates, you can simplify your maintenance work for many container workloads. Applying patches regularly and keep up-to-date in small environments is very easy.
You don't even need to deploy any additional workload to take care of these updates. The very good systemd support in Podman is doing the heavy lifting for you.
It would be nice to know if you consider to have auto-updates for your containers.