operational deployment

.omo/run-continuation/ses_f6341e990ffeuDX0xnVFWQQ8fS.json

@@ -0,0 +1,10 @@
+{
+  "sessionID": "ses_f6341e990ffeuDX0xnVFWQQ8fS",
+  "updatedAt": "2026-09-13T22:17:31.684Z",
+  "sources": {
+    "background-task": {
+      "state": "idle",
+      "updatedAt": "2026-09-13T22:17:31.684Z"
+    }
+  }
+}

.omo/run-continuation/ses_f63e23ebcffe7Q6sJKCl9W8sFt.json

@@ -1,10 +1,10 @@
 {
   "sessionID": "ses_f63e23ebcffe7Q6sJKCl9W8sFt",
-  "updatedAt": "2026-09-13T19:22:32.998Z",
+  "updatedAt": "2026-09-13T21:32:58.318Z",
   "sources": {
     "background-task": {
       "state": "idle",
-      "updatedAt": "2026-09-13T19:22:32.998Z"
+      "updatedAt": "2026-09-13T21:32:58.318Z"
     }
   }
 }

README.md

@@ -1,6 +1,6 @@
 # Stoat — Fedora 44 self-hosted chat deployment
 
-Ansible automation that **hardens a Fedora 44 server**, installs **rootless Podman**, and deploys a self-hosted **[Stoat](https://stoat.chat)** instance (open-source community chat, formerly Revolt) using the official [`stoatchat/self-hosted`](https://github.com/stoatchat/self-hosted) stack.
+Ansible automation that **hardens a Fedora 44 server**, installs **Podman**, and deploys a self-hosted **[Stoat](https://stoat.chat)** instance (open-source community chat, formerly Revolt) using the official [`stoatchat/self-hosted`](https://github.com/stoatchat/self-hosted) stack.
 
 **Target host:** `stoat-demo`
 
@@ -17,12 +17,11 @@ Ansible automation that **hardens a Fedora 44 server**, installs **rootless Podm
     │   ├── main.yml             # imports, in order
     │   ├── system.yml           # dnf full update + base packages
     │   ├── harden.yml           # firewalld, sshd, fail2ban, dnf5-automatic, logrotate
-    │   ├── podman.yml           # rootless podman, compose, service user, linger
+    │   ├── podman.yml           # podman, compose, SELinux container policy
     │   └── stoat.yml            # clone repo → generate config → systemd unit → start
     ├── handlers/main.yml
     └── templates/
         ├── jail.local.j2                    # fail2ban config
-        ├── sshd_hardening.conf.j2           # sshd drop-in
         └── podman-compose-stoat.service.j2  # systemd unit for the stack
 ```
 
@@ -44,14 +43,10 @@ Ansible automation that **hardens a Fedora 44 server**, installs **rootless Podm
    stoat-demo ansible_host=stoat-demo ansible_user=YOUR_SSH_USER
    ```
 
-3. **SSH password login is intentionally enabled** (`sshd_password_authentication: yes`).
-   - Root login stays disabled — use your SSH user + `sudo`.
-   - Fail2Ban is the brute-force defense: `3` failed attempts → `1 h` ban (tunable).
-   - If you ever want key-only auth again, flip the var to `no`.
-   - Ansible connects with a password via `-k`:
+3. **SSH is not touched by this playbook** — the server keeps its SSH configuration exactly as you have it. Fail2Ban still protects the login (`3` failed attempts → `1 h` ban, tunable). Ansible connects with your SSH password via `-k`:
 
    ```bash
-   ansible-playbook -i inventory.ini site.yml -k
+   ansible-playbook -i inventory.ini site.yml
    ```
 
 ---
@@ -61,12 +56,12 @@ Ansible automation that **hardens a Fedora 44 server**, installs **rootless Podm
 | Stage | Actions |
 |---|---|
 | **System** | `dnf update` all packages; install `git`, `curl`, `openssl` |
-| **Firewall** | firewalld default-deny; allow `ssh`, `http`, `https`, `7881/tcp`, `50000-50100/udp` (LiveKit voice) |
-| **SSH** | drop-in `00-hardening.conf`: `PermitRootLogin no`, `PasswordAuthentication yes`; config validated with `sshd -t` before restart |
+| **Firewall** | installs + starts firewalld (native/minimal Fedora may not ship it); default-deny; allow `ssh`, `http`, `https`, `7881/tcp`, `50000-50100/udp` (LiveKit voice) |
 | **Fail2Ban** | `sshd` jail (3 retries → 1 h ban, incremental for repeat offenders) |
 | **Auto-updates** | `dnf5-plugin-automatic`, security-only, `apply_updates = yes`, started timer |
 | **Logs** | logrotate rule for fail2ban logs (daily, 7 rotations, compressed) |
-| **Podman** | rootless engine + `podman-compose` + `podman-docker`; `net.ipv4.ip_unprivileged_port_start=80` so Caddy can bind 80/443; dedicated `stoat` service user with lingering enabled |
+| **Podman** | engine + `podman-compose` + `podman-docker`; runs as root (no unprivileged service user, no linger) |
+| **SELinux** | keeps mode enforcing; installs `container-selinux` (container policy) + `policycoreutils-python-utils` / `setools-console` (`semanage`, `audit2why`); registers the deployment tree as `container_file_t` so rootful containers can use the bind mounts |
 | **Stoat** | clone `stoatchat/self-hosted` to `/opt/stoat/self-hosted`; run `generate_config.sh <domain>` (answers: built-in Caddy, video enabled); start stack under `podman-compose-stoat.service` |
 
 ---
@@ -91,12 +86,8 @@ Ansible automation that **hardens a Fedora 44 server**, installs **rootless Podm
 | `stoat_version` | `main` | git ref of `self-hosted` repo |
 | `stoat_reverse_proxy` | `false` | `true` = move Caddy to `:8880`, bring your own front proxy/TLS |
 | `stoat_video` | `true` | enable camera/screen sharing (LiveKit) |
-| `stoat_service_user` | `stoat` | unprivileged user running rootless podman |
-| `podman_unprivileged_port_start` | `80` | lowest port rootless containers may bind |
+| `selinux_mode` | `enforcing` | SELinux mode the role enforces (config + setenforce) |
 | `fail2ban_port` / `fail2ban_maxretry` / `fail2ban_bantime` | `22` / `3` / `3600` | sshd jail tuning |
-| `sshd_permit_root_login` | `no` | root SSH access |
-| `sshd_password_authentication` | `yes` | password login (fail2ban guards it) |
-| `sshd_mfa_required` | `false` | only enable if a 2FA PAM module exists, else you lock yourself out |
 | `firewall_services` / `firewall_ports` | ssh,http,https / 7881, UDP range | firewalld allow rules |
 
 Override any variable per-run, without editing files:
@@ -121,7 +112,7 @@ ansible-playbook -i inventory.ini site.yml -k -e "stoat_domain=chat.example.org
 
 ## Day-to-day operations
 
-All commands run on the server, mostly as the `stoat` user (`sudo -u stoat` when needed):
+All commands run on the server as root:
 
 ```bash
 # Stack status
@@ -129,14 +120,14 @@ systemctl status podman-compose-stoat
 podman ps
 
 # Logs (per service: api / events / web / livekit / database …)
-sudo -u stoat podman logs -f api
+podman logs -f api
 
 # Restart the whole stack
-sudo systemctl restart podman-compose-stoat
+systemctl restart podman-compose-stoat
 
-# Update Stoat to new images (run in /opt/stoat/self-hosted as stoat)
-sudo -u stoat podman-compose pull
-sudo -u stoat podman-compose up -d
+# Update Stoat to new images (run in /opt/stoat/self-hosted)
+podman-compose pull
+podman-compose up -d
 
 # Fail2Ban status
 sudo fail2ban-client status sshd
@@ -147,11 +138,38 @@ sudo firewall-cmd --list-all
 
 ---
 
+## SELinux notes
+
+Fedora 44 ships SELinux **enforcing**; the role pins that mode. Containers run as root under the `container_t` domain, so the role registers `/opt/stoat/self-hosted` as **`container_file_t`** (via `semanage fcontext` + `restorecon`) — that's what lets the rootful containers read/write the bind-mounted `data/` volumes and config files without per-volume `:Z` flags, and the registration is persistent (survives `restorecon`).
+
+If a service misbehaves, check for denials before anything else:
+
+```bash
+sudo ausearch -m avc -ts recent | grep -i stoat    # recent denials
+# or find the exact rule suggestion:
+sudo grep "denied" /var/log/audit/audit.log | tail -20 | audit2why
+```
+
+Common knobs if a denial shows up (only set what the denial actually requires):
+
+```bash
+# containers connecting to non-default ports on the host
+sudo setsebool -P container_connect_any 1
+# container needs to manage cgroups (systemd inside a container)
+sudo setsebool -P container_manage_cgroup 1
+# re-apply file contexts if you add bind mounts outside the
+# deployment tree (the role only labels /opt/stoat/self-hosted):
+sudo semanage fcontext -a -t container_file_t '/opt/stoat/moredata(/.*)?'
+sudo restorecon -RF /opt/stoat/moredata
+```
+
+---
+
 ## Caveats & notes
 
 - **First start pulls ~2 GB of images** and initializes MongoDB — allow a few minutes; the systemd unit tolerates up to 15 min (`TimeoutStartSec=900`).
-- **Podman, not Docker**: the upstream stack is docker-compose based; this deployment runs it with `podman-compose` (rootless). It supports the compose features this stack needs (`depends_on: condition: service_healthy`, bind mounts, port ranges).
-- **`generate_config.sh` is guarded** by `creates:` — it never overwrites existing `secrets.env`/`Revolt.toml` unintentionally. To regenerate config deliberately, run it manually with `--overwrite` as the `stoat` user (after backing up secrets).
-- **Security posture**: rootless containers, non-root SSH user, fail2ban, firewalld default-deny, security-only auto-updates. (Aligned with the upstream guide's recommendations.)
-- **Rootless bind mounts**: data persists in `/opt/stoat/self-hosted/data/` (mongodb, rabbitmq, minio, caddy).
+- **Podman, not Docker**: the upstream stack is docker-compose based; this deployment runs it with `podman-compose` as root. It supports the compose features this stack needs (`depends_on: condition: service_healthy`, bind mounts, port ranges).
+- **`generate_config.sh` is guarded** by `creates:` — it never overwrites existing `secrets.env`/`Revolt.toml` unintentionally. To regenerate config deliberately, run it manually with `--overwrite` (after backing up secrets).
+- **Security posture**: fail2ban, firewalld default-deny, security-only auto-updates, SELinux enforcing with `container_file_t` labels. SSH configuration is left exactly as you set it. (Aligned with the upstream guide's recommendations.)
+- **Data persistence**: everything persists in `/opt/stoat/self-hosted/data/` (mongodb, rabbitmq, minio, caddy) — keep it out of the git clone/reset.
 - System hardening + Stoat are one role run; to re-apply only config changes, re-running `site.yml` is idempotent.

group_vars/all/vars.yml

@@ -33,20 +33,13 @@ stoat_reverse_proxy: false
 stoat_video: true
 
 # ------------------------------------------------------------
-# Podman (rootless)
+# Podman (runs as root)
 # ------------------------------------------------------------
-# Unprivileged system user that runs the rootless podman stack
-stoat_service_user: chatmanone
-
 podman_packages:
   - podman
   - podman-compose          # compose file runner (supports service_healthy conditions)
   - podman-docker           # `docker` CLI compatibility, backed by podman
 
-# Rootless containers cannot bind ports < 1024 by default.
-# Caddy needs 80/443, so lower the unprivileged port floor.
-podman_unprivileged_port_start: 80
-
 # ------------------------------------------------------------
 # Hardening - Fail2Ban
 # ------------------------------------------------------------
@@ -67,6 +60,14 @@ sshd_password_authentication: yes
 # out. Leave false unless you know what you are doing.
 sshd_mfa_required: false
 
+# ------------------------------------------------------------
+# Hardening - SELinux
+# ------------------------------------------------------------
+# Native Fedora 44 ships SELinux enforcing; the role keeps it in
+# this mode and installs container-selinux + policycoreutils
+# (semanage / audit2why) for diagnosing container denials.
+selinux_mode: enforcing
+
 # ------------------------------------------------------------
 # Hardening - firewall (firewalld, default deny)
 # ------------------------------------------------------------

inventory.ini

@@ -1,15 +1,16 @@
 # --------------------------------------------------------------
 # Ansible inventory - servers to harden
-# Usage: ansible-playbook -i inventory.ini harden_fedora44.yml
+# Usage: ansible-playbook -i inventory.ini site.yml -k
 # --------------------------------------------------------------
 
 [servers]
-stoat-demo ansible_host=stoat-demo
+# Replace YOUR_SSH_PASSWORD with the real password. Keep the file
+# out of git / chmod 600 it, or encrypt with ansible-vault:
+#   ansible-vault encrypt inventory.ini
+stoat-demo
 
 [servers:vars]
 # Elevate privileges (the playbook already sets become: true,
 # but this allows per-group override if needed)
 ansible_become=yes
 ansible_become_method=sudo
-# Optional: path to your SSH private key
-# ansible_ssh_private_key_file=/home/you/.ssh/id_ed25519

roles/stoat/tasks/harden.yml

@@ -1,7 +1,14 @@
 ---
 # ------------------------------------------------------------
 # Firewall (firewalld) - default deny, allow only what is needed
+# Native/minimal Fedora installs may not ship firewalld, so
+# install it explicitly before starting it.
 # ------------------------------------------------------------
+- name: Install firewalld
+  ansible.builtin.dnf:
+    name: firewalld
+    state: present
+
 - name: Start and enable firewalld
   ansible.builtin.systemd:
     name: firewalld
@@ -34,29 +41,6 @@
   when: item.rc | int != 0
   notify: Reload firewalld
 
-# ------------------------------------------------------------
-# SSH hardening (Drop-in config, validated before restart)
-# ------------------------------------------------------------
-- name: Deploy sshd hardening drop-in
-  ansible.builtin.template:
-    src: sshd_hardening.conf.j2
-    dest: /etc/ssh/sshd_config.d/00-hardening.conf
-    owner: root
-    group: root
-    mode: "0644"
-  register: sshd_conf
-
-- name: Validate sshd configuration
-  ansible.builtin.command: sshd -t
-  changed_when: false
-  when: sshd_conf is changed
-
-- name: Restart sshd
-  ansible.builtin.service:
-    name: sshd
-    state: restarted
-  when: sshd_conf is changed
-
 # ------------------------------------------------------------
 # Fail2Ban - brute force protection
 # ------------------------------------------------------------
@@ -88,19 +72,24 @@
     name: dnf5-plugin-automatic
     state: present
 
+# ini_file with create: true handles both a missing config file
+# (package may not ship /etc/dnf/automatic.conf until a run) and
+# preserves any other sections the user may have customized.
 - name: Enable automatic installation of updates
-  ansible.builtin.lineinfile:
+  community.general.ini_file:
     path: /etc/dnf/automatic.conf
-    insertafter: "^\\[commands\\]"
-    regexp: "^#?\\s*apply_updates\\s*="
-    line: "apply_updates = yes"
+    section: commands
+    option: apply_updates
+    value: "yes"
+    create: true
 
 - name: Restrict automatic updates to security errata
-  ansible.builtin.lineinfile:
+  community.general.ini_file:
     path: /etc/dnf/automatic.conf
-    insertafter: "^\\[commands\\]"
-    regexp: "^#?\\s*upgrade_type\\s*="
-    line: "upgrade_type = security"
+    section: commands
+    option: upgrade_type
+    value: "security"
+    create: true
 
 - name: Enable and start dnf5-automatic.timer
   ansible.builtin.systemd:

roles/stoat/tasks/main.yml

@@ -2,7 +2,7 @@
 # stoat role - task order matters:
 #   system  -> update + baseline packages (git needed by stoat tasks)
 #   harden  -> firewall, ssh, fail2ban, automatic updates, logrotate
-#   podman  -> rootless podman + service user
+#   podman  -> podman + compose + SELinux container policy
 #   stoat   -> clone repo, generate config, systemd-managed compose stack
 - name: System update and base packages
   ansible.builtin.import_tasks: system.yml
@@ -10,7 +10,7 @@
 - name: Harden the host
   ansible.builtin.import_tasks: harden.yml
 
-- name: Install rootless podman
+- name: Install podman
   ansible.builtin.import_tasks: podman.yml
 
 - name: Deploy Stoat

roles/stoat/tasks/podman.yml

@@ -1,33 +1,38 @@
 ---
 # ------------------------------------------------------------
-# Podman - rootless engine, compose support, unprivileged user
+# Podman - engine, compose support (runs as root, no service user)
 # ------------------------------------------------------------
 - name: Install podman stack
   ansible.builtin.dnf:
     name: "{{ podman_packages }}"
     state: present
 
-# Rootless podman cannot bind host ports < 1024 without this.
-# Caddy (Stoat's TLS reverse proxy) needs 80/443.
-- name: Allow rootless containers to bind privileged ports
-  ansible.builtin.sysctl:
-    name: net.ipv4.ip_unprivileged_port_start
-    value: "{{ podman_unprivileged_port_start }}"
-    sysctl_file: /etc/sysctl.d/99-podman.conf
+# ------------------------------------------------------------
+# SELinux - keep enforcing, ensure the container policy and
+# diagnostic tools are present for resolving any denials.
+# ------------------------------------------------------------
+- name: Install SELinux container policy and diagnostics tools
+  ansible.builtin.dnf:
+    name:
+      - container-selinux
+      - policycoreutils-python-utils
+      - setools-console
     state: present
-    reload: true
 
-- name: Create dedicated service user for rootless podman
-  ansible.builtin.user:
-    name: "{{ stoat_service_user }}"
-    system: true
-    shell: /bin/bash
-    create_home: true
-    comment: "Stoat rootless podman service user"
-  register: stoat_system_user
-
-# Keeps /run/user/<uid> alive and lets rootless containers start
-# on boot without an interactive login.
-- name: Enable lingering for the service user
-  ansible.builtin.command: "loginctl enable-linger {{ stoat_service_user }}"
+- name: Check current SELinux mode
+  ansible.builtin.command: getenforce
+  register: selinux_current_mode
+  changed_when: false
+
+- name: Configure SELinux mode in /etc/selinux/config
+  ansible.builtin.lineinfile:
+    path: /etc/selinux/config
+    regexp: '^SELINUX='
+    line: 'SELINUX={{ selinux_mode }}'
+
+- name: Apply SELinux mode now
+  ansible.builtin.command: "setenforce {{ selinux_mode }}"
+  when:
+    - selinux_mode | lower == "enforcing"
+    - selinux_current_mode.stdout | lower != "enforcing"
   changed_when: false

roles/stoat/tasks/stoat.yml

@@ -1,35 +1,56 @@
 ---
 # ------------------------------------------------------------
-# Stoat deployment - rootless podman-compose, managed by systemd
+# Stoat deployment - podman-compose (as root), managed by systemd
 # ------------------------------------------------------------
+# The whole stack runs as root (play-level become): the repo and
+# config are generated as root and the systemd unit runs
+# podman-compose without dropping privileges.
+# The deployment dir may already be owned by another user from
+# earlier runs; register it so git-as-root accepts it.
+- name: Allow git to operate on the deployment directory
+  ansible.builtin.command:
+    cmd: "git config --global --add safe.directory {{ stoat_install_dir }}"
+  changed_when: false
+
 - name: Clone Stoat self-hosted repository
   ansible.builtin.git:
     repo: "{{ stoat_repo }}"
     dest: "{{ stoat_install_dir }}"
     version: "{{ stoat_version }}"
     force: true
-  become: true
-  become_user: "{{ stoat_service_user }}"
-  become_method: su
-  environment:
-    HOME: "/home/{{ stoat_service_user }}"
 
 - name: Ensure generate_config.sh is executable
   ansible.builtin.file:
     path: "{{ stoat_install_dir }}/generate_config.sh"
     mode: "0755"
-  become: true
-  become_user: "{{ stoat_service_user }}"
-  become_method: su
+
+# Upstream compose.yml still references docker.io/minio/minio and
+# docker.io/minio/mc, which no longer exist on Docker Hub (the pulls are
+# denied, so `podman-compose up` hangs and api/events/etc. never start).
+# MinIO's official images live on quay.io - rewrite the refs.
+- name: Point MinIO images at quay.io (docker.io/minio/* is gone)
+  ansible.builtin.replace:
+    path: "{{ stoat_install_dir }}/compose.yml"
+    regexp: 'docker\.io/minio/(minio|mc)'
+    replace: 'quay.io/minio/\1'
+
+# autumn depends on createbuckets, a one-shot that exits 0 after creating
+# the MinIO buckets. podman-compose treats `condition: service_started` as
+# "container is running", so it waits forever on the exited one-shot and
+# `podman-compose up` never completes (autumn/pushd stay Created). Use the
+# service_completed_successfully condition instead.
+- name: Fix autumn dependency on the createbuckets one-shot
+  ansible.builtin.replace:
+    path: "{{ stoat_install_dir }}/compose.yml"
+    regexp: '(createbuckets:\n\s+condition: )service_started'
+    replace: '\1service_completed_successfully'
 
 # The compose file bind-mounts these directories; create them up
-# front so rootless podman does not fail on missing sources.
+# front so podman-compose does not fail on missing sources.
 - name: Create data directories referenced by compose bind mounts
   ansible.builtin.file:
     path: "{{ stoat_install_dir }}/data/{{ item }}"
     state: directory
-    owner: "{{ stoat_service_user }}"
-    group: "{{ stoat_service_user }}"
     mode: "0755"
   loop:
     - db
@@ -50,11 +71,27 @@
   args:
     chdir: "{{ stoat_install_dir }}"
     creates: "{{ stoat_install_dir }}/Revolt.toml"
-  become: true
-  become_user: "{{ stoat_service_user }}"
-  become_method: su
-  environment:
-    HOME: "/home/{{ stoat_service_user }}"
+
+# Rootful podman containers run as container_t; on enforcing
+# SELinux they cannot read/write bind mounts labeled usr_t, so
+# register the deployment tree as container_file_t (persistent).
+- name: Ensure the deployment tree is root-owned
+  ansible.builtin.file:
+    path: "{{ stoat_install_dir }}"
+    state: directory
+    owner: root
+    group: root
+    recurse: true
+
+- name: Register the deployment directory as a container file context
+  community.general.sefcontext:
+    target: "{{ stoat_install_dir }}(/.*)?"
+    setype: container_file_t
+    state: present
+
+- name: Relabel the deployment tree with restorecon
+  ansible.builtin.command: "restorecon -RF {{ stoat_install_dir }}"
+  changed_when: false
 
 - name: Deploy podman-compose systemd unit
   ansible.builtin.template:
@@ -65,11 +102,10 @@
     mode: "0644"
 
 # First start pulls ~2 GB of images + initializes MongoDB, which can
-# take several minutes. TimeoutStartSec covers this inside the unit.
+# take several minutes. TimeoutStartSec in the unit covers this.
 - name: Enable and start the Stoat service (first pull may take minutes)
   ansible.builtin.systemd:
     name: podman-compose-stoat
     state: started
     enabled: true
     daemon_reload: true
-    timeout: 900

roles/stoat/templates/podman-compose-stoat.service.j2

@@ -6,15 +6,10 @@ Wants=network-online.target
 
 [Service]
 # oneshot + RemainAfterExit: the unit tracks compose state while
-# the containers themselves run detached under rootless podman.
+# the containers themselves run detached under the root podman.
 Type=oneshot
 RemainAfterExit=yes
-User={{ stoat_service_user }}
-Group={{ stoat_service_user }}
 WorkingDirectory={{ stoat_install_dir }}
-# Rootless podman needs the per-user runtime directory; %U expands
-# to the UID of `User=`. Lingering (enable-linger) keeps it alive.
-Environment=XDG_RUNTIME_DIR=/run/user/%U
 ExecStart=/usr/bin/podman-compose up -d --remove-orphans
 ExecStop=/usr/bin/podman-compose down
 # First start pulls ~2 GB of images; be generous.

roles/stoat/templates/sshd_hardening.conf.j2

@@ -1,8 +0,0 @@
-# Managed by Ansible - stoat role (sshd hardening drop-in)
-# Values come from group_vars/all/vars.yml
-
-PermitRootLogin {{ sshd_permit_root_login }}
-PasswordAuthentication {{ sshd_password_authentication }}
-{% if sshd_mfa_required %}
-AuthenticationMethods publickey,keyboard-interactive:pam
-{% endif %}

site.yml

@@ -1,7 +1,7 @@
 ---
 # ============================================================
 # Stoat.chat Fedora 44 server
-# Harden the host, install rootless Podman, deploy Stoat.
+# Harden the host, install Podman, deploy Stoat.
 #
 # Usage:
 #   ansible-playbook -i inventory.ini site.yml