Podman Quadlet

In the past, I used deploy services with docker-compose. usually, It works well. But my os is Fedora, the firewall is firewalld, docker always break the firewall rules by iptables. and then docker needs root permission to run, so I want to find a better way to run my services. So I found podman and quadlet.

Fundamentally, Docker Compose also offers process supervision similar to the host’s systemd. Quadlet inherits systemd’s benefits and can reuse the existing toolchain, so logs and state management don’t need a new toolchain, significantly lowering the learning curve — though its configuration differs from systemd and took some time to adapt to.

So migrating from docker compose to quadlet is not easy. During that time I tried to use Podman Compose directly to simplify my migration, but compatibility was poor, so I ended up making major changes to my deployment configuration.

The first service i migrated is my file browser service which is a service to offer the record of live files for my friend. I have a docker-compose file like this:

config

version: '3.8'

services:
  biligo:
    image: chigusa/bililive-go:v0.7.25
    restart: unless-stopped
    volumes:
      - /mnt/study/biligo:/srv/bililive
      - ./config:/etc/bililive-go
    ports:
      - target: 8080
        published: 8888
    networks:
      - bililive
    environment:
      - PUID=1000
      - PGID=1000
      - UMASK=022
  alist:
    restart: always
    volumes:
      - './alist/config:/opt/alist/data'
      - '/mnt/study/biligo:/data:ro'
    ports:
      - '5244:5244'
    environment:
      - PUID=1000
      - PGID=1000
      - UMASK=022
    container_name: alist
    image: 'xhofe/alist:latest'
    networks:
      - bililive
  #file-server:
  #  image: nginx:1.23
  #  restart: unless-stopped
  #  volumes:
  #    - /mnt/study/biligo:/srv:ro
  #    - ${PWD}/nginx-config/default.conf:/etc/nginx/conf.d/default.conf
  #    - ${PWD}/nginx-config/autoindex.xslt:/etc/nginx/autoindex.xslt
  #    - ${PWD}/nginx-config/nginx.conf:/etc/nginx/nginx.conf
  #  ports:
  #    - 192.168.35.1:9000:80
  #  networks:
  #    - bililive
  reverse-proxy:
    image: caddy/cloudflare
    build:
      dockerfile: ./caddy-file
      context: .
    restart: unless-stopped
    user: '1000:1000'
    environment:
      - CLOUDFLARE_API_TOKEN=secret
    volumes:
      - ${PWD}/caddy/Caddyfile:/etc/caddy/Caddyfile
      - ${PWD}/caddy/data:/data
      - ${PWD}/caddy/config:/config
    ports:
      - 9999:5244
      - 9999:5244/udp
    networks:
      - bililive

networks:
  bililive:
    driver_opts:
      com.docker.network.bridge.name: br-bililive
    name: br-bililive

At first I used Nginx to provide download services for my friends, but it couldn’t preview files. Later I switched to Alist to serve files externally, which solved the problem, so I commented out the Nginx configuration.

However, if I migrate to Quadlet, I’ll need to create around ten files: container configurations for Alist, Caddy, and bililive-go; the configuration files and data mounts for each service; and the pod and network definitions.

One difference compared with Docker Compose is that these containers run inside a single pod — that is, within the same cgroup environment — so communication between them can go over the loopback interface instead of the full network stack (though I don’t really need that performance improvement). Docker Compose’s DNS already handled the container domain names, so I hadn’t considered this approach before. Docker probably can achieve the same, but I haven’t looked into it in detail.

Here are the quadlet files I created for the services:

# /etc/containers/systemd/bili/bili.pod
[Pod]
PodName=bili
PodmanArgs=--userns auto:uidmapping=0:1000:1000,gidmapping=0:1000:1000
Network=bili.network
PublishPort=8888:8080
PublishPort=9999:5244
PublishPort=9999:5244/udp

# /etc/containers/systemd/bili/bili.network
# To maintain compatibility with Docker, Podman also supports some of Docker's configuration names. :)
[Network]
NetworkName=bili
Options=com.docker.network.bridge.name=br-bili

# /etc/containers/systemd/bili/caddy.container
[Unit]
Description=caddy
After=network-online.target

[Container]
Image=localhost/caddy-cloudflare
SecurityLabelDisable=true
NoNewPrivileges=true
EnvironmentFile=/home/user/config/bililive-go/config/caddy/env
Mount=type=bind,src=/home/user/config/bililive-go/config/caddy/Caddyfile,target=/etc/caddy/Caddyfile
Volume=caddy-data.volume:/data
Volume=caddy-config.volume:/config
StopTimeout=30
Pod=bili.pod

[Service]
Restart=always
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

# /etc/containers/systemd/bili/caddy-data.volume

[Volume]
Device=/home/user/config/bililive-go/data/caddy
VolumeName=caddy-data
Options=rbind
Driver=local
Type=none

The container syntax is very similar to systemd’s, and services can have dependency relationships; for the exact syntax, refer to Quadlet’s documentation.

Other configurations omitted for brevity.

systemd

After creating these files, we should reload systemd configurations:

sudo systemctl daemon-reload

and then you can find the generated service files in /run/systemd/generator (fedora 42):

for example, there is the generated service file for pod:

# Automatically generated by /usr/lib/systemd/system-generators/podman-system-generator
# bili-pod.service
[X-Pod]
PodName=bili
PodmanArgs=--userns auto:uidmapping=0:1000:1000,gidmapping=0:1000:1000
Network=bili.network
PublishPort=8888:8080
PublishPort=9999:5244
PublishPort=9999:5244/udp

[Unit]
Wants=network-online.target
After=network-online.target
SourcePath=/etc/containers/systemd/bili/bili.pod
RequiresMountsFor=%t/containers
Wants=alist.service
Before=alist.service
Wants=caddy.service
Before=caddy.service
Wants=bili.service
Before=bili.service
Requires=bili-network.service
After=bili-network.service

[Service]
SyslogIdentifier=%N
ExecStart=/usr/bin/podman pod start bili
ExecStop=/usr/bin/podman pod stop --ignore --time=10 bili
ExecStopPost=/usr/bin/podman pod rm --ignore --force bili
ExecStartPre=/usr/bin/podman pod create --infra-conmon-pidfile=%t/%N.pid --replace --exit-policy stop --publish 8888:8080 --publish 9999:5244 --publish 9999:5244/udp --network bili --infra-name bili-infra --name bili --userns auto:uidmapping=0:1000:1000,gidmapping=0:1000:1000
Environment=PODMAN_SYSTEMD_UNIT=%n
Type=forking
Restart=on-failure
PIDFile=%t/%N.pid

# Automatically generated by /usr/lib/systemd/system-generators/podman-system-generator
# bili.service
[Unit]
Wants=network-online.target
After=network-online.target
Description=bililive-go
After=network-online.target
SourcePath=/etc/containers/systemd/bili/bili.container
RequiresMountsFor=%t/containers
Requires=bililive-data-volume.service
After=bililive-data-volume.service
Requires=bililive-config-volume.service
After=bililive-config-volume.service
BindsTo=bili-pod.service
After=bili-pod.service

[X-Container]
Image=docker.io/chigusa/bililive-go:v0.7.38
SecurityLabelDisable=true
NoNewPrivileges=true
Volume=bililive-data.volume:/srv/bililive
Volume=bililive-config.volume:/etc/bililive-go
StopTimeout=30
Pod=bili.pod

[Service]
Restart=always
SuccessExitStatus=143
Environment=PODMAN_SYSTEMD_UNIT=%n
KillMode=mixed
ExecStop=/usr/bin/podman rm -v -f -i systemd-%N
ExecStopPost=-/usr/bin/podman rm -v -f -i systemd-%N
Delegate=yes
Type=notify
NotifyAccess=all
SyslogIdentifier=%N
ExecStart=/usr/bin/podman run --name systemd-%N --replace --rm --cgroups=split --stop-timeout 30 --sdnotify=conmon -d --security-opt=no-new-privileges --security-opt label=disable -v bililive-data:/srv/bililive -v bililive-config:/etc/bililive-go --pod bili docker.io/chigusa/bililive-go:v0.7.38

[Install]
WantedBy=multi-user.target

Now we can start the pod with:

sudo systemctl start bili.pod

congratulations! you have migrated your services from docker-compose to podman-quadlet. You can use nearly all of systemd’s functionality, and its default security mechanisms help prevent resource abuse.

However, running multiple containers in a single pod can cause UID-mapping and SELinux permission issues. For UID mapping, check whether it creates permission problems; if so, consider changing the container runtime UID or using User=xxx to synchronize users across containers in the pod. For SELinux, if you need convenience or are debugging, I recommend adding SecurityLabelDisable=true to disable SELinux labeling inside the containers. If you want SELinux to protect your data and system, create custom SELinux policies; avoid changing the host SELinux labels for certain system files (for example files under /etc), as Podman also advises against that. Choose the approach that best fits your needs.For more details, please refer to the Podman documentation.

logs

because quadlet use systemd to manage the services, so you can use journalctl to view the logs:

journalctl -u bili.service -f