From 41d9283a73b6846285206a502b828225e6ee33bf Mon Sep 17 00:00:00 2001 From: modem7 Date: Fri, 23 Oct 2026 12:13:44 +0100 Subject: [PATCH 1/3] Builder: provider-neutral mail relay (gmail, custom, none) Adds MAIL_RELAY and related settings. GMAIL_USER and GMAIL_APP_PASSWORD still work as the old names. --- .../Ubuntu Cloud Template/README.md | 36 ++- .../Ubuntu Cloud Template/build-templates.sh | 228 +++++++++++++++--- .../Ubuntu Cloud Template/shared.conf.example | 16 +- 3 files changed, 230 insertions(+), 50 deletions(-) diff --git a/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md b/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md index 098a55e..61a7aa2 100644 --- a/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md +++ b/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md @@ -36,7 +36,7 @@ are committed. `create-ubuntu-cloud-template.sh --config` (see `profile.conf.example`). No secrets. - `shared.conf`: the opinionated side, the same for every template. Ansible - key, Gmail relay, unattended upgrades mail, dotfiles, WakaTime, login + key, mail relay, unattended upgrades mail, dotfiles, WakaTime, login shell, needrestart, Pro adverts and the journald cap. Also a block of host defaults (`DISTRO_VER`, `DISK_STOR`, `NET_BRIDGE`, `VLAN`, `CI_SSH_KEY`, `CLOUD_PASSWORD`) used by every template unless its profile sets them. @@ -91,7 +91,7 @@ a config file or profiles. The default arguments passed to the generic script are `--unattended --vmid 52000 --auto-vmid --template`. -`shared.conf` holds the Gmail app password and SSH keys. It is git-ignored +`shared.conf` holds the mail password and SSH keys. It is git-ignored (`*.conf`). Never commit it. ### Moving from `build-templates.conf` @@ -109,17 +109,37 @@ repo on every run (falling back to the existing copy). ## Mail on clones -Each clone relays mail through `[smtp.gmail.com]:587` using the Gmail app -password from the config. +Each clone sends mail through the relay chosen by `MAIL_RELAY` in +`shared.conf`: + +| `MAIL_RELAY` | What it does | +| --- | --- | +| `gmail` (default) | `[smtp.gmail.com]:587` with `MAIL_RELAY_USER` and a Gmail app password in `MAIL_RELAY_PASSWORD`. The only one tested. | +| `custom` | Your own relay: `MAIL_RELAY_HOST` and `MAIL_RELAY_PORT`. Port 587 uses STARTTLS (required), 465 uses TLS from the start, anything else (such as 25) uses TLS if offered. A blank `MAIL_RELAY_USER` means no login, for an internal relay. | +| `none` | No relay. Mail stays on the clone, and anything addressed outside it bounces. | + +Other settings: `MAIL_ROOT_TO` (where root's mail goes; defaults to +`MAIL_RELAY_USER` when that is an address), `MAIL_SENDER` (optional: rewrite +every sender to this address, which some relays require) and `MAIL_DOMAIN`. +Only the `gmail` relay has been tested; `custom` and `none` are checked by +reading the generated config, not by sending mail. The builder refuses the +placeholder values from `shared.conf.example`. + +`GMAIL_USER` and `GMAIL_APP_PASSWORD` still work as the old names for +`MAIL_RELAY_USER` and `MAIL_RELAY_PASSWORD`, with a warning. + +`verify-template.sh` and `verify-clone.sh` still expect the Gmail relay, so +for `custom` or `none` expect FAILs in their mail section until they read +`shared.conf`. - The brackets stop MX lookups. -- IPv4 only, TLS required, and queued mail expires after one day, so a failure - defers and drops instead of looping. +- IPv4 only, TLS where the relay supports it, and queued mail expires after + one day, so a failure defers and drops instead of looping. - A per-boot script sets `/etc/mailname` and `myhostname` from the clone's own hostname plus `MAIL_DOMAIN`, so clones never keep the template's name. - Mail is sent as the natural `user@hostname.domain`. There is no sender or - `From:` rewrite. -- `root` mail is aliased to `GMAIL_USER`. + `From:` rewrite unless `MAIL_SENDER` is set. +- `root` mail is aliased to `MAIL_ROOT_TO`. - `/etc/crontab` sets `MAILTO=""`. Users' own crontabs are not changed, so add `MAILTO=""` to those if you do not want cron mail. diff --git a/Bash/Proxmox Scripts/Ubuntu Cloud Template/build-templates.sh b/Bash/Proxmox Scripts/Ubuntu Cloud Template/build-templates.sh index 0f031a7..4134411 100755 --- a/Bash/Proxmox Scripts/Ubuntu Cloud Template/build-templates.sh +++ b/Bash/Proxmox Scripts/Ubuntu Cloud Template/build-templates.sh @@ -63,9 +63,16 @@ CLOUD_PASSWORD="" # blank = generated each build, shown at the ANSIBLE_SSH_PUBKEY="" # required -GMAIL_USER="" # required -GMAIL_APP_PASSWORD="" # required +MAIL_RELAY="gmail" # gmail, custom (MAIL_RELAY_HOST/PORT) or none (local only) +MAIL_RELAY_HOST="" # custom only +MAIL_RELAY_PORT="" # custom only: 587, 465 or 25 +MAIL_RELAY_USER="" # SMTP login; required for gmail, blank (custom) = no login +MAIL_RELAY_PASSWORD="" # required when MAIL_RELAY_USER is set +MAIL_ROOT_TO="" # root's mail; blank = MAIL_RELAY_USER if it has an @ +MAIL_SENDER="" # blank = no sender rewrite MAIL_DOMAIN="" # required; mail is sent as user@.MAIL_DOMAIN +GMAIL_USER="" # old names for MAIL_RELAY_USER / MAIL_RELAY_PASSWORD +GMAIL_APP_PASSWORD="" UU_MAIL="" # blank = no unattended-upgrades mail @@ -90,7 +97,8 @@ JOURNAL_MAX_PERCENT="10" # a profile can't break Postfix, unattended upgrades or the login shell. # bsd-mailx is installed by the postfix payload, AFTER the debconf preseed: # installed earlier it would pull postfix in with the default answers. -BUILDER_PKGS=(libsasl2-modules unattended-upgrades zsh) +# libsasl2-modules is added after shared.conf is read, if the relay logs in. +BUILDER_PKGS=(unattended-upgrades zsh) # ============================================================================= # HELP @@ -132,10 +140,10 @@ TEMPLATES unzip. desktop ubuntu-2604-desktop. extras plus GNOME and RDP. Keeps snapd. 4096 MB RAM, 40G disk. - All three get: Postfix relay via Gmail, an ansible user with sudo, - unattended upgrades and zram to zswap on first boot. Also, each as set - in shared.conf: a journald size cap, dotfiles in /etc/skel, and the - login shell, needrestart and Pro advert tweaks. + All three get: Postfix (relay set by MAIL_RELAY), an ansible user with + sudo, unattended upgrades and zram to zswap on first boot. Also, each + as set in shared.conf: a journald size cap, dotfiles in /etc/skel, and + the login shell, needrestart and Pro advert tweaks. CONFIG Two kinds of file, both git-ignored. Copy the .example next to each. @@ -162,15 +170,36 @@ CONFIG 2. profiles/.conf as it is, so it can override step 1. Override the SSH key there with SSH_KEY, not CI_SSH_KEY. 3. The payloads, appended with +=: UPLOAD_FILES, CUSTOMIZE_SCRIPTS, - FIRSTBOOT_SCRIPTS, EXTRA_VC_ARGS, and any of libsasl2-modules, - unattended-upgrades and zsh missing from EXTRA_VIRT_PKGS. + FIRSTBOOT_SCRIPTS, EXTRA_VC_ARGS, and any of unattended-upgrades, + zsh and (if the relay logs in) libsasl2-modules missing from + EXTRA_VIRT_PKGS. shared.conf, required: ANSIBLE_SSH_PUBKEY Public key for the ansible user. - GMAIL_USER Gmail account mail is relayed through. - GMAIL_APP_PASSWORD App password for that account. MAIL_DOMAIN Mail is sent as user@.MAIL_DOMAIN. + shared.conf, mail (default): + MAIL_RELAY gmail. One of: + gmail smtp.gmail.com port 587, with a login + custom MAIL_RELAY_HOST and MAIL_RELAY_PORT + none no relay: mail stays on the clone + MAIL_RELAY_HOST custom only: host name or IPv4 address. + MAIL_RELAY_PORT custom only: 587 (STARTTLS, required), 465 + (TLS from the start), or 25 or any other + port (TLS if offered). + MAIL_RELAY_USER SMTP login. Required for gmail. For custom, + blank = no login (an internal relay). + MAIL_RELAY_PASSWORD Password for that login (Gmail: an app + password). Required when MAIL_RELAY_USER is set. + MAIL_ROOT_TO Where root's mail goes. Default: MAIL_RELAY_USER + if it is an address, else required (with none, + blank = /etc/aliases left alone). + MAIL_SENDER blank = no rewrite: mail goes out as + user@. Set = every sender rewritten to + this address (some relays need it). + GMAIL_USER and GMAIL_APP_PASSWORD are still read as the old names + for MAIL_RELAY_USER and MAIL_RELAY_PASSWORD, with a warning. + shared.conf, host defaults (default): DISTRO_VER resolute DISK_STOR blank = the host's only VM-disk storage @@ -268,6 +297,24 @@ mapfile -t leftover < <(compgen -v PKGS_ || true) || warn "Ignored in $SHARED_CONF (set EXTRA_VIRT_PKGS and TAG in profiles/.conf instead): ${leftover[*]}" unset leftover +# Mail: old Gmail names, relay presets, root alias. Checked in generate(). +if [[ -n "$GMAIL_USER" || -n "$GMAIL_APP_PASSWORD" ]]; then + warn "GMAIL_USER / GMAIL_APP_PASSWORD are old names: rename them to MAIL_RELAY_USER / MAIL_RELAY_PASSWORD in $SHARED_CONF." + [[ -n "$MAIL_RELAY_USER" ]] || MAIL_RELAY_USER="$GMAIL_USER" + [[ -n "$MAIL_RELAY_PASSWORD" ]] || MAIL_RELAY_PASSWORD="$GMAIL_APP_PASSWORD" +fi +MAIL_RELAY="${MAIL_RELAY:-gmail}" +if [[ "$MAIL_RELAY" == gmail ]]; then + MAIL_RELAY_HOST="smtp.gmail.com" + MAIL_RELAY_PORT="587" +fi +if [[ "$MAIL_RELAY" != none && -z "$MAIL_ROOT_TO" && "$MAIL_RELAY_USER" == *@* ]]; then + MAIL_ROOT_TO="$MAIL_RELAY_USER" +fi +# True if the relay logs in (SASL). +relay_login() { [[ "$MAIL_RELAY" != none && -n "$MAIL_RELAY_USER" ]]; } +relay_login && BUILDER_PKGS=(libsasl2-modules "${BUILDER_PKGS[@]}") + # Quote a value so it can be 'source'd back safely. q() { printf '%q' "$1"; } @@ -301,13 +348,54 @@ EOF } > "$OUT_DIR/payload/ansible.sh" } +# The body is put together here, per MAIL_RELAY, so the script on the +# image only holds the steps it runs. Settings are quoted variables at the +# top; host and port are checked in generate() before they are written. +# SC2016: the single-quoted $ is payload text, expanded on the image. +# shellcheck disable=SC2016 write_postfix_payload() { + local relay="" pc=() i + [[ "$MAIL_RELAY" == none ]] || relay="[$MAIL_RELAY_HOST]:$MAIL_RELAY_PORT" + + # postconf -e arguments, as payload text. + if [[ -n "$relay" ]]; then + pc+=('"relayhost=$relay"') + case "$MAIL_RELAY_PORT" in + 587) pc+=('"smtp_tls_security_level=encrypt"') ;; + 465) pc+=('"smtp_tls_security_level=encrypt"' '"smtp_tls_wrappermode=yes"') ;; + *) pc+=('"smtp_tls_security_level=may"') ;; + esac + pc+=('"smtp_tls_CAfile=/etc/ssl/certs/ca-certificates.crt"') + fi + if relay_login; then + pc+=('"smtp_sasl_auth_enable=yes"' + '"smtp_sasl_security_options=noanonymous"' + '"smtp_sasl_password_maps=$db:/etc/postfix/sasl/sasl_passwd"') + fi + if [[ -n "$MAIL_SENDER" ]]; then + pc+=('"sender_canonical_classes=envelope_sender, header_sender"' + '"sender_canonical_maps=static:$MAIL_SENDER"') + fi + pc+=('"inet_interfaces=loopback-only"' + '"inet_protocols=ipv4"' + '"mynetworks=127.0.0.0/8"' + '"mydomain=$MAIL_DOMAIN"' + '"myhostname=$fqdn"' + '"myorigin=/etc/mailname"' + "'mydestination=\$myhostname, localhost.\$mydomain, localhost'" + '"maximal_queue_lifetime=1d"' + '"bounce_queue_lifetime=1d"') + { echo '#!/bin/bash' - echo "# Generated by build-templates.sh: postfix relay via Gmail." - echo "GMAIL_USER=$(q "$GMAIL_USER")" - echo "GMAIL_APP_PASSWORD=$(q "$GMAIL_APP_PASSWORD")" + echo "# Generated by build-templates.sh: postfix, MAIL_RELAY=$MAIL_RELAY." + if relay_login; then + echo "MAIL_RELAY_USER=$(q "$MAIL_RELAY_USER")" + echo "MAIL_RELAY_PASSWORD=$(q "$MAIL_RELAY_PASSWORD")" + fi echo "MAIL_DOMAIN=$(q "$MAIL_DOMAIN")" + [[ -z "$MAIL_ROOT_TO" ]] || echo "MAIL_ROOT_TO=$(q "$MAIL_ROOT_TO")" + [[ -z "$MAIL_SENDER" ]] || echo "MAIL_SENDER=$(q "$MAIL_SENDER")" cat <<'EOF' set -euo pipefail export DEBIAN_FRONTEND=noninteractive @@ -316,41 +404,54 @@ echo "postfix postfix/main_mailer_type select Local only" | debconf-set-selectio echo "postfix postfix/mailname string localhost" | debconf-set-selections apt-get install -y -q postfix bsd-mailx -relay="[smtp.gmail.com]:587" -db="$(postconf -h default_database_type)" # hash or lmdb +EOF + [[ -z "$relay" ]] || echo "relay=\"$relay\"" + relay_login && echo 'db="$(postconf -h default_database_type)" # hash or lmdb' + cat <<'EOF' # Placeholder until first boot: no clone is called localhost, so # postfix-identity always re-stamps this with the clone's real name. fqdn="localhost.${MAIL_DOMAIN}" echo "$fqdn" > /etc/mailname +EOF + if relay_login; then + cat <<'EOF' install -d -m 755 /etc/postfix/sasl -( umask 077; printf '%s %s:%s\n' "$relay" "$GMAIL_USER" "$GMAIL_APP_PASSWORD" > /etc/postfix/sasl/sasl_passwd ) +( umask 077; printf '%s %s:%s\n' "$relay" "$MAIL_RELAY_USER" "$MAIL_RELAY_PASSWORD" > /etc/postfix/sasl/sasl_passwd ) postmap "$db:/etc/postfix/sasl/sasl_passwd" chmod 600 /etc/postfix/sasl/sasl_passwd* -# No sender rewrite: mail goes out as the natural user@. -# Short queue lifetimes so an undeliverable message can't loop for days. -postconf -e \ - "relayhost=$relay" \ - "smtp_tls_security_level=encrypt" \ - "smtp_tls_CAfile=/etc/ssl/certs/ca-certificates.crt" \ - "smtp_sasl_auth_enable=yes" \ - "smtp_sasl_security_options=noanonymous" \ - "smtp_sasl_password_maps=$db:/etc/postfix/sasl/sasl_passwd" \ - "inet_interfaces=loopback-only" \ - "inet_protocols=ipv4" \ - "mynetworks=127.0.0.0/8" \ - "mydomain=$MAIL_DOMAIN" \ - "myhostname=$fqdn" \ - "myorigin=/etc/mailname" \ - 'mydestination=$myhostname, localhost.$mydomain, localhost' \ - "maximal_queue_lifetime=1d" \ - "bounce_queue_lifetime=1d" +EOF + fi + if [[ -n "$MAIL_SENDER" ]]; then + echo '# Every sender (envelope and From header) is rewritten to MAIL_SENDER.' + else + echo '# No sender rewrite: mail goes out as the natural user@.' + fi + echo "# Short queue lifetimes so an undeliverable message can't loop for days." + echo "postconf -e \\" + for (( i = 0; i < ${#pc[@]} - 1; i++ )); do + echo " ${pc[i]} \\" + done + echo " ${pc[-1]}" + if [[ -n "$relay" ]]; then + cat <<'EOF' # "Local only" sets these to error, which bounces every outgoing mail. postconf -X default_transport relay_transport - -# Root's local mail (bounces, users' cron jobs) goes to Gmail. -grep -q '^root:' /etc/aliases 2>/dev/null || printf 'root: %s\n' "$GMAIL_USER" >> /etc/aliases +EOF + else + cat <<'EOF' +# MAIL_RELAY=none: "Local only" stays as it is, so mail never leaves. +EOF + fi + echo + if [[ -n "$MAIL_ROOT_TO" ]]; then + cat <<'EOF' +# Root's local mail (bounces, users' cron jobs) goes to MAIL_ROOT_TO. +grep -q '^root:' /etc/aliases 2>/dev/null || printf 'root: %s\n' "$MAIL_ROOT_TO" >> /etc/aliases +EOF + fi + cat <<'EOF' newaliases # The chroot copy of resolv.conf must be root-owned, or postfix warns. @@ -631,11 +732,60 @@ check_profiles() { done } +# True if an address is one of the .example placeholders. +placeholder_addr() { + local a="${1,,}" + [[ "$a" == you@gmail.com || "$a" == you@example.com || "$a" =~ @example\.[^@]*$ ]] +} + +# The mail settings must make sense for MAIL_RELAY, with no placeholders +# left from shared.conf.example. Never prints the password. +check_mail() { + local v + case "$MAIL_RELAY" in + gmail|custom|none) ;; + *) die "MAIL_RELAY must be gmail, custom or none, not '$MAIL_RELAY'." ;; + esac + [[ ! "${MAIL_DOMAIN,,}" =~ ^example\.(com|org|net)$ ]] \ + || die "MAIL_DOMAIN is still the placeholder $MAIL_DOMAIN: set your own domain in $SHARED_CONF." + # The relay login first: MAIL_ROOT_TO may have been copied from it. + if [[ "$MAIL_RELAY" != none ]] && placeholder_addr "$MAIL_RELAY_USER"; then + die "MAIL_RELAY_USER is still the placeholder $MAIL_RELAY_USER: set your own login in $SHARED_CONF." + fi + for v in MAIL_ROOT_TO MAIL_SENDER; do + [[ -z "${!v}" || "${!v}" == *@* ]] || die "$v must be an email address." + if placeholder_addr "${!v}"; then + die "$v is still the placeholder ${!v}: set your own address in $SHARED_CONF." + fi + done + [[ "$MAIL_RELAY" == none ]] && return 0 + + if [[ "$MAIL_RELAY" == custom ]]; then + [[ -n "$MAIL_RELAY_HOST" ]] || die "Set MAIL_RELAY_HOST in $SHARED_CONF (MAIL_RELAY=custom)." + [[ "$MAIL_RELAY_HOST" =~ ^[A-Za-z0-9.-]+$ ]] \ + || die "MAIL_RELAY_HOST must be a host name or IPv4 address, not '$MAIL_RELAY_HOST'." + [[ -n "$MAIL_RELAY_PORT" ]] || die "Set MAIL_RELAY_PORT in $SHARED_CONF (MAIL_RELAY=custom)." + if [[ ! "$MAIL_RELAY_PORT" =~ ^[1-9][0-9]{0,4}$ ]] || (( MAIL_RELAY_PORT > 65535 )); then + die "MAIL_RELAY_PORT must be a port number, not '$MAIL_RELAY_PORT'." + fi + else + [[ -n "$MAIL_RELAY_USER" ]] || die "Set MAIL_RELAY_USER in $SHARED_CONF (MAIL_RELAY=gmail)." + fi + if [[ -n "$MAIL_RELAY_USER" ]]; then + [[ -n "$MAIL_RELAY_PASSWORD" ]] || die "Set MAIL_RELAY_PASSWORD in $SHARED_CONF (MAIL_RELAY_USER is set)." + [[ ! "$MAIL_RELAY_PASSWORD" =~ ^x+$ ]] \ + || die "MAIL_RELAY_PASSWORD is still the xxxx placeholder: set the real password in $SHARED_CONF." + fi + [[ -n "$MAIL_ROOT_TO" ]] \ + || die "Set MAIL_ROOT_TO in $SHARED_CONF: where root's mail goes (MAIL_RELAY_USER is not an address)." +} + generate() { local v - for v in ANSIBLE_SSH_PUBKEY GMAIL_USER GMAIL_APP_PASSWORD MAIL_DOMAIN; do + for v in ANSIBLE_SSH_PUBKEY MAIL_DOMAIN; do [[ -n "${!v}" ]] || die "Set $v in $SHARED_CONF." done + check_mail [[ -z "$JOURNAL_MAX_MB" || ( "$JOURNAL_MAX_MB" =~ ^[0-9]+$ && "$JOURNAL_MAX_PERCENT" =~ ^[0-9]+$ ) ]] \ || die "JOURNAL_MAX_MB and JOURNAL_MAX_PERCENT must be numbers." [[ -z "$NEEDRESTART_MODE" || "$NEEDRESTART_MODE" =~ ^[ali]$ ]] || die "NEEDRESTART_MODE must be a, l, i or blank." diff --git a/Bash/Proxmox Scripts/Ubuntu Cloud Template/shared.conf.example b/Bash/Proxmox Scripts/Ubuntu Cloud Template/shared.conf.example index ebc6e12..d7dc7c9 100644 --- a/Bash/Proxmox Scripts/Ubuntu Cloud Template/shared.conf.example +++ b/Bash/Proxmox Scripts/Ubuntu Cloud Template/shared.conf.example @@ -30,10 +30,20 @@ CLOUD_PASSWORD="" # blank = generated each build # --- Ansible user (required) --- ANSIBLE_SSH_PUBKEY="ssh-ed25519 AAAA... ansible" -# --- Postfix relay through Gmail (required) --- -GMAIL_USER="you@gmail.com" -GMAIL_APP_PASSWORD="xxxxxxxxxxxxxxxx" # Google app password +# --- Postfix mail relay --- +# MAIL_RELAY: gmail = smtp.gmail.com port 587 (MAIL_RELAY_HOST/PORT ignored) +# custom = MAIL_RELAY_HOST and MAIL_RELAY_PORT below +# none = no relay: mail stays on the clone +MAIL_RELAY="gmail" +MAIL_RELAY_HOST="" # custom only: host name or IPv4 address +MAIL_RELAY_PORT="" # custom only: 587 (STARTTLS, required), + # 465 (TLS from the start), 25 (TLS if offered) +MAIL_RELAY_USER="you@gmail.com" # SMTP login; custom: blank = no login +MAIL_RELAY_PASSWORD="xxxxxxxxxxxxxxxx" # Gmail: a Google app password +MAIL_ROOT_TO="" # root's mail; blank = MAIL_RELAY_USER if it is an address +MAIL_SENDER="" # blank = no rewrite; set = every sender becomes this MAIL_DOMAIN="example.com" # mail is sent as user@.example.com +# The old names GMAIL_USER and GMAIL_APP_PASSWORD still work (with a warning). # --- Unattended upgrades report (blank = no mail) --- UU_MAIL="you@gmail.com" From 29c4e7658acb7c973a6da9aea29e52efb334385e Mon Sep 17 00:00:00 2001 From: modem7 Date: Fri, 23 Oct 2026 12:15:49 +0100 Subject: [PATCH 2/3] Builder: require TLS on any custom relay port when there is a login A login on a port other than 587 or 465 used smtp_tls_security_level=may, so the password could be sent in plain text if TLS failed. It is now encrypt whenever a login is set. Opportunistic TLS stays for a no-login relay. --- Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md | 2 +- .../Ubuntu Cloud Template/build-templates.sh | 7 +++++-- .../Ubuntu Cloud Template/shared.conf.example | 3 ++- 3 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md b/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md index 61a7aa2..b5d8bb1 100644 --- a/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md +++ b/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md @@ -115,7 +115,7 @@ Each clone sends mail through the relay chosen by `MAIL_RELAY` in | `MAIL_RELAY` | What it does | | --- | --- | | `gmail` (default) | `[smtp.gmail.com]:587` with `MAIL_RELAY_USER` and a Gmail app password in `MAIL_RELAY_PASSWORD`. The only one tested. | -| `custom` | Your own relay: `MAIL_RELAY_HOST` and `MAIL_RELAY_PORT`. Port 587 uses STARTTLS (required), 465 uses TLS from the start, anything else (such as 25) uses TLS if offered. A blank `MAIL_RELAY_USER` means no login, for an internal relay. | +| `custom` | Your own relay: `MAIL_RELAY_HOST` and `MAIL_RELAY_PORT`. Port 587 uses STARTTLS (required), 465 uses TLS from the start, anything else (such as 25) requires TLS when there is a login and uses it if offered otherwise. A blank `MAIL_RELAY_USER` means no login, for an internal relay. | | `none` | No relay. Mail stays on the clone, and anything addressed outside it bounces. | Other settings: `MAIL_ROOT_TO` (where root's mail goes; defaults to diff --git a/Bash/Proxmox Scripts/Ubuntu Cloud Template/build-templates.sh b/Bash/Proxmox Scripts/Ubuntu Cloud Template/build-templates.sh index 4134411..d006878 100755 --- a/Bash/Proxmox Scripts/Ubuntu Cloud Template/build-templates.sh +++ b/Bash/Proxmox Scripts/Ubuntu Cloud Template/build-templates.sh @@ -186,7 +186,8 @@ CONFIG MAIL_RELAY_HOST custom only: host name or IPv4 address. MAIL_RELAY_PORT custom only: 587 (STARTTLS, required), 465 (TLS from the start), or 25 or any other - port (TLS if offered). + port (TLS required if there is a login, + otherwise TLS if offered). MAIL_RELAY_USER SMTP login. Required for gmail. For custom, blank = no login (an internal relay). MAIL_RELAY_PASSWORD Password for that login (Gmail: an app @@ -363,7 +364,9 @@ write_postfix_payload() { case "$MAIL_RELAY_PORT" in 587) pc+=('"smtp_tls_security_level=encrypt"') ;; 465) pc+=('"smtp_tls_security_level=encrypt"' '"smtp_tls_wrappermode=yes"') ;; - *) pc+=('"smtp_tls_security_level=may"') ;; + *) # With a login, never fall back to plain text. + if relay_login; then pc+=('"smtp_tls_security_level=encrypt"') + else pc+=('"smtp_tls_security_level=may"'); fi ;; esac pc+=('"smtp_tls_CAfile=/etc/ssl/certs/ca-certificates.crt"') fi diff --git a/Bash/Proxmox Scripts/Ubuntu Cloud Template/shared.conf.example b/Bash/Proxmox Scripts/Ubuntu Cloud Template/shared.conf.example index d7dc7c9..f8a6081 100644 --- a/Bash/Proxmox Scripts/Ubuntu Cloud Template/shared.conf.example +++ b/Bash/Proxmox Scripts/Ubuntu Cloud Template/shared.conf.example @@ -37,7 +37,8 @@ ANSIBLE_SSH_PUBKEY="ssh-ed25519 AAAA... ansible" MAIL_RELAY="gmail" MAIL_RELAY_HOST="" # custom only: host name or IPv4 address MAIL_RELAY_PORT="" # custom only: 587 (STARTTLS, required), - # 465 (TLS from the start), 25 (TLS if offered) + # 465 (TLS from the start), 25 (TLS required with a login, + # else if offered) MAIL_RELAY_USER="you@gmail.com" # SMTP login; custom: blank = no login MAIL_RELAY_PASSWORD="xxxxxxxxxxxxxxxx" # Gmail: a Google app password MAIL_ROOT_TO="" # root's mail; blank = MAIL_RELAY_USER if it is an address From a8893aef5d2fb0d2e678bc90dc409c234f4bbc4c Mon Sep 17 00:00:00 2001 From: modem7 Date: Fri, 23 Oct 2026 12:25:41 +0100 Subject: [PATCH 3/3] Verify: read expected values from shared.conf, support custom and none relays verify-clone.sh sources shared.conf on the host and passes only non-secret values to the guest as VERIFY_* variables. verify-template.sh keeps its old defaults when they are not set. --- .../Ubuntu Cloud Template/README.md | 32 +-- .../Ubuntu Cloud Template/verify-clone.sh | 104 +++++++++- .../Ubuntu Cloud Template/verify-template.sh | 185 +++++++++++++----- 3 files changed, 261 insertions(+), 60 deletions(-) diff --git a/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md b/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md index 61a7aa2..e5c5aef 100644 --- a/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md +++ b/Bash/Proxmox Scripts/Ubuntu Cloud Template/README.md @@ -115,7 +115,7 @@ Each clone sends mail through the relay chosen by `MAIL_RELAY` in | `MAIL_RELAY` | What it does | | --- | --- | | `gmail` (default) | `[smtp.gmail.com]:587` with `MAIL_RELAY_USER` and a Gmail app password in `MAIL_RELAY_PASSWORD`. The only one tested. | -| `custom` | Your own relay: `MAIL_RELAY_HOST` and `MAIL_RELAY_PORT`. Port 587 uses STARTTLS (required), 465 uses TLS from the start, anything else (such as 25) uses TLS if offered. A blank `MAIL_RELAY_USER` means no login, for an internal relay. | +| `custom` | Your own relay: `MAIL_RELAY_HOST` and `MAIL_RELAY_PORT`. Port 587 uses STARTTLS (required), 465 uses TLS from the start. Any other port (such as 25) requires TLS when there is a login, and uses TLS if offered when there is none. A blank `MAIL_RELAY_USER` means no login, for an internal relay. | | `none` | No relay. Mail stays on the clone, and anything addressed outside it bounces. | Other settings: `MAIL_ROOT_TO` (where root's mail goes; defaults to @@ -128,13 +128,9 @@ placeholder values from `shared.conf.example`. `GMAIL_USER` and `GMAIL_APP_PASSWORD` still work as the old names for `MAIL_RELAY_USER` and `MAIL_RELAY_PASSWORD`, with a warning. -`verify-template.sh` and `verify-clone.sh` still expect the Gmail relay, so -for `custom` or `none` expect FAILs in their mail section until they read -`shared.conf`. - - The brackets stop MX lookups. -- IPv4 only, TLS where the relay supports it, and queued mail expires after - one day, so a failure defers and drops instead of looping. +- IPv4 only, TLS (required on 587, 465 or with a login), and queued mail + expires after one day, so a failure defers and drops instead of looping. - A per-boot script sets `/etc/mailname` and `myhostname` from the clone's own hostname plus `MAIL_DOMAIN`, so clones never keep the template's name. - Mail is sent as the natural `user@hostname.domain`. There is no sender or @@ -160,7 +156,9 @@ Options (see `--help` for all of them): - `--profile` and `--send-test-mail
` are passed to `verify-template.sh`. With the mail option the script also waits up to 60 - seconds for Postfix to log `status=sent` for that address. + seconds for Postfix to log `status=sent` for that address. With + `MAIL_RELAY=none` it does not wait (mail stays on the clone). +- `--config `: the `shared.conf` to read (see below). - `--vmid `: VMID for the clone. Default: the next free one. A VMID that already exists is refused. - `--full [--storage ]`: full clone. Default is a linked clone, which @@ -169,6 +167,16 @@ Options (see `--help` for all of them): - `--timeout ` and `--verify-timeout `: waits for the agent and cloud-init (default 300), and for the verify run (default 600). +What to expect comes from `shared.conf`, found as `build-templates.sh` finds +it: `--config`, then `SHARED_CONF`, then `BUILD_CONF`, then `shared.conf` next +to the script. A file named by one of the first three must exist. The file is +read in a subshell, and only non-secret settings go to the guest, as +`VERIFY_*` variables: the mail relay, host, port and login name, `MAIL_ROOT_TO`, +`MAIL_SENDER`, `MAIL_DOMAIN`, `UU_MAIL`, the ansible key, `LOGIN_SHELL`, +`NEEDRESTART_MODE` and the journal cap. The relay password and the other +secrets are never read out or sent. With no `shared.conf` found, it says so and +`verify-template.sh` uses its built-in defaults. + The template needs `qemu-guest-agent` installed and the VM option `agent=1`. The clone is destroyed on success, failure and Ctrl-C. Only a VM this run created is ever stopped or destroyed. Exit codes: 0 passed, 1 a FAIL or a @@ -191,9 +199,11 @@ Options: The exit code is 0 when there are no FAILs (warnings are fine). -`verify-template.sh` has the values it expects (domain, Gmail user, SSH key, -time zone and so on) at the top. Edit them to match your `shared.conf` and -profiles. +Run by hand, `verify-template.sh` uses the values at its top (mail relay, +domain, relay login, SSH key, time zone and so on). Edit them to match your +`shared.conf` and profiles, or use `verify-clone.sh`, which passes them from +`shared.conf`. Settings left blank in `shared.conf` (no relay login, no +`UU_MAIL`, no `LOGIN_SHELL` and so on) are shown as `SKIP` or checked as off. ## Manual steps diff --git a/Bash/Proxmox Scripts/Ubuntu Cloud Template/verify-clone.sh b/Bash/Proxmox Scripts/Ubuntu Cloud Template/verify-clone.sh index fc4b525..55368a2 100755 --- a/Bash/Proxmox Scripts/Ubuntu Cloud Template/verify-clone.sh +++ b/Bash/Proxmox Scripts/Ubuntu Cloud Template/verify-clone.sh @@ -7,6 +7,10 @@ # it, boots the clone, waits for the guest agent and cloud-init, runs # verify-template.sh inside it through the agent, then destroys the clone. # +# The values to expect come from shared.conf (found as build-templates.sh +# finds it). Only non-secret settings are passed to the guest, never a +# password. No shared.conf: verify-template.sh's own defaults are used. +# # Usage (as root, on the Proxmox host): # ./verify-clone.sh [options] # ./verify-clone.sh --help # full help, no root needed @@ -42,6 +46,9 @@ Clones the template, starts the clone, waits for the guest agent and for cloud-init to finish, runs verify-template.sh inside it through the guest agent, prints the result, then destroys the clone. +What to expect (mail relay, domain, ansible key, shell, and so on) is +read from shared.conf, so the check matches what the builder made. + USAGE ./verify-clone.sh [options] ./verify-clone.sh -h|--help|help @@ -52,7 +59,11 @@ ARGUMENTS verify-template.sh. Default: it works it out. --send-test-mail Passed to verify-template.sh, which sends a test mail. This script then waits up to 60s - for Postfix to log status=sent for it. + for Postfix to log status=sent for it + (not with MAIL_RELAY=none: mail stays local). + --config shared.conf to read. Default: SHARED_CONF, + then BUILD_CONF, then shared.conf next to + this script. --vmid VMID for the clone. Must not exist yet. Default: the next free one (pvesh nextid). --full Full clone. Default: linked clone. @@ -82,6 +93,17 @@ EXAMPLES ./verify-clone.sh 9002 --send-test-mail you@example.com ./verify-clone.sh 9002 --full --storage local-lvm --vmid 9101 ./verify-clone.sh 9002 --keep + ./verify-clone.sh 9002 --config /root/other.conf + +SHARED.CONF + Read the same way as build-templates.sh, with the same defaults and + old names (GMAIL_USER). Only these are passed to the guest, as VERIFY_* + variables: MAIL_RELAY, MAIL_RELAY_HOST, MAIL_RELAY_PORT, + MAIL_RELAY_USER, MAIL_ROOT_TO, MAIL_SENDER, MAIL_DOMAIN, UU_MAIL, + ANSIBLE_SSH_PUBKEY, LOGIN_SHELL, NEEDRESTART_MODE, JOURNAL_MAX_MB and + JOURNAL_MAX_PERCENT. No password or other secret is read out or sent. + A file named by --config, SHARED_CONF or BUILD_CONF must exist. With no + shared.conf next to this script, verify-template.sh's defaults apply. NOTES - Run as root on the Proxmox host. @@ -97,7 +119,7 @@ EOF # ============================================================================= # ARGUMENTS # ============================================================================= -TEMPLATE="" PROFILE="" TEST_MAIL="" NEWID="" FULL=0 STORAGE="" KEEP=0 +TEMPLATE="" PROFILE="" TEST_MAIL="" NEWID="" FULL=0 STORAGE="" KEEP=0 CONFIG="" TIMEOUT=300 VERIFY_TIMEOUT=600 VERIFY_SCRIPT="$SCRIPT_DIR/verify-template.sh" usage_error() { { echo "Error: $*"; echo "Run ./verify-clone.sh --help for usage."; } >&2; exit 2; } @@ -115,6 +137,7 @@ while [[ $# -gt 0 ]]; do --timeout) need_val "$@"; TIMEOUT="$2"; shift 2 ;; --verify-timeout) need_val "$@"; VERIFY_TIMEOUT="$2"; shift 2 ;; --script) need_val "$@"; VERIFY_SCRIPT="$2"; shift 2 ;; + --config) need_val "$@"; CONFIG="$2"; shift 2 ;; -*) usage_error "unknown option: $1" ;; *) [[ -z "$TEMPLATE" ]] || usage_error "more than one template VMID given." TEMPLATE="$1"; shift ;; @@ -158,6 +181,73 @@ grep -q '^# verify-template.sh$' "$VERIFY_SCRIPT" \ size=$(stat -c %s "$VERIFY_SCRIPT") (( size <= MAX_STDIN )) || die "$VERIFY_SCRIPT is $size bytes: qm guest exec takes at most $MAX_STDIN on stdin." 2 +# ============================================================================= +# SHARED CONFIG +# Found as build-templates.sh finds it (the old build-templates.conf is not +# looked for). Sourced in a subshell that prints only the allow-listed, +# non-secret values, NUL-separated: no password reaches this shell, the +# qm command line or the guest. +# ============================================================================= +read_conf() { # : prints VERIFY_x=value NUL ..., ending with VERIFY_FROM_CONFIG=1 + ( + set +u + # The builder's defaults, so a setting left out means the same here. + MAIL_RELAY="gmail" MAIL_RELAY_HOST="" MAIL_RELAY_PORT="" MAIL_RELAY_USER="" + MAIL_ROOT_TO="" MAIL_SENDER="" MAIL_DOMAIN="" GMAIL_USER="" UU_MAIL="" + ANSIBLE_SSH_PUBKEY="" LOGIN_SHELL="/usr/bin/zsh" NEEDRESTART_MODE="a" + JOURNAL_MAX_MB="1024" JOURNAL_MAX_PERCENT="10" + # shellcheck source=/dev/null + source "$1" >/dev/null 2>&1 + unset MAIL_RELAY_PASSWORD GMAIL_APP_PASSWORD CLOUD_PASSWORD WAKATIME_CFG + # Old name, relay preset and root alias, as in build-templates.sh. + [[ -n "$MAIL_RELAY_USER" ]] || MAIL_RELAY_USER="$GMAIL_USER" + MAIL_RELAY="${MAIL_RELAY:-gmail}" + [[ "$MAIL_RELAY" == gmail ]] && { MAIL_RELAY_HOST="smtp.gmail.com"; MAIL_RELAY_PORT="587"; } + if [[ "$MAIL_RELAY" != none && -z "$MAIL_ROOT_TO" && "$MAIL_RELAY_USER" == *@* ]]; then + MAIL_ROOT_TO="$MAIL_RELAY_USER" + fi + # if_set: blank = verify-template.sh's default. always: blank = "off". + if_set() { [[ -z "$2" ]] || printf '%s=%s\0' "$1" "$2"; } + always() { printf '%s=%s\0' "$1" "$2"; } + always VERIFY_MAIL_RELAY "$MAIL_RELAY" + if_set VERIFY_RELAY_HOST "$MAIL_RELAY_HOST" + if_set VERIFY_RELAY_PORT "$MAIL_RELAY_PORT" + always VERIFY_RELAY_USER "$MAIL_RELAY_USER" + always VERIFY_ROOT_TO "$MAIL_ROOT_TO" + always VERIFY_MAIL_SENDER "$MAIL_SENDER" + if_set VERIFY_MAIL_DOMAIN "$MAIL_DOMAIN" + always VERIFY_UU_MAIL "$UU_MAIL" + if_set VERIFY_ANSIBLE_KEY "$ANSIBLE_SSH_PUBKEY" + always VERIFY_SHELL "$LOGIN_SHELL" + always VERIFY_NEEDRESTART "$NEEDRESTART_MODE" + always VERIFY_JOURNAL_MAX_MB "$JOURNAL_MAX_MB" + if_set VERIFY_JOURNAL_PERCENT "$JOURNAL_MAX_PERCENT" + printf 'VERIFY_FROM_CONFIG=1\0' + ) +} + +if [[ -n "$CONFIG" ]]; then CONF_FILE="$CONFIG" +elif [[ -n "${SHARED_CONF:-}" ]]; then CONF_FILE="$SHARED_CONF" +elif [[ -n "${BUILD_CONF:-}" ]]; then CONF_FILE="$BUILD_CONF" +elif [[ -f "$SCRIPT_DIR/shared.conf" ]]; then CONF_FILE="$SCRIPT_DIR/shared.conf" +else CONF_FILE=""; fi + +conf_env=() RELAY="" +if [[ -n "$CONF_FILE" ]]; then + [[ -f "$CONF_FILE" && -r "$CONF_FILE" ]] || die "shared.conf not found or not readable: $CONF_FILE" 2 + # Output dropped: an error message could quote a line holding a password. + bash -n "$CONF_FILE" &>/dev/null || die "$CONF_FILE has a syntax error (check it with: bash -n)." 2 + mapfile -d '' -t conf_env < <(read_conf "$CONF_FILE") + (( ${#conf_env[@]} )) && [[ "${conf_env[-1]}" == VERIFY_FROM_CONFIG=1 ]] \ + || die "could not read $CONF_FILE." 2 + for kv in "${conf_env[@]}"; do [[ "$kv" == VERIFY_MAIL_RELAY=* ]] && RELAY="${kv#*=}"; done + [[ "$RELAY" =~ ^(gmail|custom|none)$ ]] \ + || die "MAIL_RELAY in $CONF_FILE must be gmail, custom or none, not '$RELAY'." 2 + info "Expected values from $CONF_FILE (non-secret settings only)" +else + info "No shared.conf found: verify-template.sh's built-in defaults apply (or use --config )." +fi + tpl_conf="$("$QM" config "$TEMPLATE" 2>&1)" || die "VM $TEMPLATE not found: $tpl_conf" 2 grep -qx 'template: 1' <<< "$tpl_conf" || die "VM $TEMPLATE is not a template." 2 grep -qE '^agent: *(1|enabled=1)' <<< "$tpl_conf" \ @@ -329,16 +419,22 @@ esac verify_args=() [[ -n "$PROFILE" ]] && verify_args+=(--profile "$PROFILE") [[ -n "$TEST_MAIL" ]] && verify_args+=(--send-test-mail "$TEST_MAIL") +# The expected values go in as env arguments: qm passes each one as its +# own argument, so no shell quoting is involved. +guest_cmd=(/bin/bash -s -- "${verify_args[@]}") +(( ${#conf_env[@]} )) && guest_cmd=(env "${conf_env[@]}" "${guest_cmd[@]}") info "Running $(basename "$VERIFY_SCRIPT") in $NEWID (up to ${VERIFY_TIMEOUT}s)" echo -e "${B}----- guest output -----${N}" -guest_exec "$NEWID" "$VERIFY_TIMEOUT" "$VERIFY_SCRIPT" -- /bin/bash -s -- "${verify_args[@]}" +guest_exec "$NEWID" "$VERIFY_TIMEOUT" "$VERIFY_SCRIPT" -- "${guest_cmd[@]}" RC=$? echo -e "${B}----- end of guest output -----${N}" (( RC == 5 )) && exit 5 # Postfix sends in the background: wait for the relay's verdict. -if [[ -n "$TEST_MAIL" ]]; then +if [[ -n "$TEST_MAIL" && "$RELAY" == none ]]; then + echo -e " ${Y}WARN${N} not waiting for the test mail: MAIL_RELAY=none keeps mail on the clone" +elif [[ -n "$TEST_MAIL" ]]; then info "Waiting for Postfix to relay the test mail (up to ${MAIL_WAIT}s)" mail_deadline=$((SECONDS + MAIL_WAIT)) while :; do diff --git a/Bash/Proxmox Scripts/Ubuntu Cloud Template/verify-template.sh b/Bash/Proxmox Scripts/Ubuntu Cloud Template/verify-template.sh index 4ddd6c8..82c9605 100755 --- a/Bash/Proxmox Scripts/Ubuntu Cloud Template/verify-template.sh +++ b/Bash/Proxmox Scripts/Ubuntu Cloud Template/verify-template.sh @@ -13,25 +13,35 @@ # --send-test-mail Also sends the postfix test email (the one manual step). # # Exit code: 0 = no FAILs (WARNs are fine), 1 = at least one FAIL. -# Holds no secrets: it checks the Gmail password is set, never what it is. +# Holds no secrets: it checks the relay password is set, never what it is. +# +# verify-clone.sh passes the shared.conf values as VERIFY_* variables. +# Run by hand, the defaults below are used. Set but empty = that setting +# is off (checked as off, or SKIP). # ============================================================================= set -uo pipefail # --- What the templates are meant to have (match shared.conf and profiles/*.conf) --- +# ${VERIFY_x-default}: an empty VERIFY_x is kept (the setting is off). EXPECT_LANG="en_GB.UTF-8" EXPECT_TZ="Europe/London" EXPECT_KEYMAP="gb" -EXPECT_MAIL_DOMAIN="modem7.com" -EXPECT_GMAIL_USER="modem7@gmail.com" -EXPECT_UU_MAIL="modem7@gmail.com" -EXPECT_ANSIBLE_KEY="ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFv+XL86DUoq1o4EEC24uYecv4nU76XcmTeOO1rin9gx ansible" -EXPECT_SHELL="/usr/bin/zsh" -EXPECT_NEEDRESTART="a" -EXPECT_JOURNAL_MAX_MB=1024 -EXPECT_JOURNAL_PERCENT=10 +EXPECT_MAIL_DOMAIN="${VERIFY_MAIL_DOMAIN-modem7.com}" +EXPECT_RELAY="${VERIFY_MAIL_RELAY-gmail}" # gmail, custom or none +EXPECT_RELAY_HOST="${VERIFY_RELAY_HOST-smtp.gmail.com}" +EXPECT_RELAY_PORT="${VERIFY_RELAY_PORT-587}" +EXPECT_RELAY_USER="${VERIFY_RELAY_USER-modem7@gmail.com}" # empty = no relay login +EXPECT_ROOT_TO="${VERIFY_ROOT_TO-modem7@gmail.com}" # empty = no root alias +EXPECT_SENDER="${VERIFY_MAIL_SENDER-}" # empty = no sender rewrite +EXPECT_UU_MAIL="${VERIFY_UU_MAIL-modem7@gmail.com}" # empty = no upgrade mail +EXPECT_ANSIBLE_KEY="${VERIFY_ANSIBLE_KEY-ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFv+XL86DUoq1o4EEC24uYecv4nU76XcmTeOO1rin9gx ansible}" +EXPECT_SHELL="${VERIFY_SHELL-/usr/bin/zsh}" # empty = left as bash +EXPECT_NEEDRESTART="${VERIFY_NEEDRESTART-a}" # empty = OS default +EXPECT_JOURNAL_MAX_MB="${VERIFY_JOURNAL_MAX_MB-1024}" # empty = OS default +EXPECT_JOURNAL_PERCENT="${VERIFY_JOURNAL_PERCENT-10}" EXPECT_TUNED="virtual-guest" -PKGS_CORE="aptitude auto-apt-proxy libsasl2-modules tuned tuned-utils unattended-upgrades zsh +PKGS_CORE="aptitude auto-apt-proxy tuned tuned-utils unattended-upgrades zsh qemu-guest-agent cloud-init cloud-utils cloud-guest-utils postfix bsd-mailx" PKGS_TOOLS="acl bind9-dnsutils cifs-utils curl git htop needrestart net-tools plocate unzip" PKGS_DESKTOP="ubuntu-desktop-minimal gnome-remote-desktop openssl" @@ -45,17 +55,30 @@ while [[ $# -gt 0 ]]; do case "$1" in --profile) PROFILE="${2:-}"; shift 2 ;; --send-test-mail) TEST_MAIL="${2:-}"; shift 2 ;; - -h|--help) sed -n '2,16p' "$0"; exit 0 ;; + -h|--help) sed -n '2,20p' "$0"; exit 0 ;; *) echo "Unknown argument: $1" >&2; exit 2 ;; esac done +[[ "$EXPECT_RELAY" =~ ^(gmail|custom|none)$ ]] \ + || { echo "VERIFY_MAIL_RELAY must be gmail, custom or none, not '$EXPECT_RELAY'." >&2; exit 2; } +# Checked as numbers: they go into $(( )). +[[ "$EXPECT_JOURNAL_MAX_MB" =~ ^[0-9]*$ && "$EXPECT_JOURNAL_PERCENT" =~ ^[0-9]+$ ]] \ + || { echo "VERIFY_JOURNAL_MAX_MB and VERIFY_JOURNAL_PERCENT must be whole numbers." >&2; exit 2; } [[ $EUID -eq 0 ]] || { echo "Run as root (sudo $0)." >&2; exit 2; } +# The relay logs in (SASL) only with a user, as in build-templates.sh. +RELAY_LOGIN=0 +[[ "$EXPECT_RELAY" != none && -n "$EXPECT_RELAY_USER" ]] && RELAY_LOGIN=1 +(( RELAY_LOGIN )) && PKGS_CORE+=" libsasl2-modules" +LOGIN_NAME="relay login" +[[ "$EXPECT_RELAY" == gmail ]] && LOGIN_NAME="Gmail login" + if [[ -t 1 ]]; then G='\033[0;32m' Y='\033[1;33m' R='\033[0;31m' B='\033[1m' N='\033[0m'; else G='' Y='' R='' B='' N=''; fi PASSES=0 WARNS=0 FAILS=0 pass() { echo -e " ${G}PASS${N} $*"; PASSES=$((PASSES + 1)); } warn() { echo -e " ${Y}WARN${N} $*"; WARNS=$((WARNS + 1)); } fail() { echo -e " ${R}FAIL${N} $*"; FAILS=$((FAILS + 1)); } +skip() { echo -e " SKIP $*"; } # not counted: the setting is off on purpose section() { echo -e "\n${B}== $* ==${N}"; } # check "" PASS if the command succeeds check() { local d="$1"; shift; if "$@" &>/dev/null; then pass "$d"; else fail "$d"; fi; } @@ -89,6 +112,11 @@ fi echo -e "${B}Template check: $(hostname) - profile: ${PROFILE}${AUTO}${N}" echo "Login user: ${LOGIN_USER:-(none found)} Kernel: $(uname -r)" +if [[ "${VERIFY_FROM_CONFIG:-}" == 1 ]]; then + echo "Expected values: from shared.conf (mail relay: $EXPECT_RELAY)" +else + echo "Expected values: built-in defaults or VERIFY_* (mail relay: $EXPECT_RELAY)" +fi # ============================================================================= section "First boot" @@ -247,10 +275,30 @@ check "postfix running" systemctl is-active --quiet postfix check "postfix config valid (postfix check)" postfix check pc() { postconf -h "$1" 2>/dev/null; } expect_pc() { local v; v="$(pc "$1")"; [[ "$v" == "$2" ]] && pass "$1 = $v" || fail "$1 = '$v', expected '$2'"; } -expect_pc relayhost "[smtp.gmail.com]:587" -expect_pc smtp_tls_security_level encrypt -expect_pc smtp_sasl_auth_enable yes -expect_pc smtp_sasl_security_options noanonymous +if [[ "$EXPECT_RELAY" == none ]]; then + v="$(pc relayhost)" + [[ -z "$v" ]] && pass "relayhost unset: no relay (MAIL_RELAY=none), mail stays local" \ + || fail "relayhost = '$v', expected unset (MAIL_RELAY=none)" +else + expect_pc relayhost "[$EXPECT_RELAY_HOST]:$EXPECT_RELAY_PORT" + # TLS as build-templates.sh sets it: required on 587, 465 or with a login. + if [[ "$EXPECT_RELAY_PORT" == 465 ]]; then + expect_pc smtp_tls_security_level encrypt + expect_pc smtp_tls_wrappermode yes + elif [[ "$EXPECT_RELAY_PORT" == 587 ]] || (( RELAY_LOGIN )); then + expect_pc smtp_tls_security_level encrypt + else + expect_pc smtp_tls_security_level may + fi + if (( RELAY_LOGIN )); then + expect_pc smtp_sasl_auth_enable yes + expect_pc smtp_sasl_security_options noanonymous + else + v="$(pc smtp_sasl_auth_enable)" + [[ "$v" != yes ]] && pass "smtp_sasl_auth_enable = ${v:-unset} (no relay login)" \ + || fail "smtp_sasl_auth_enable = yes, but no relay login is expected" + fi +fi expect_pc inet_interfaces loopback-only expect_pc inet_protocols ipv4 expect_pc mydomain "$EXPECT_MAIL_DOMAIN" @@ -261,28 +309,53 @@ expect_pc myhostname "$fqdn" expect_pc myorigin /etc/mailname expect_pc maximal_queue_lifetime 1d expect_pc bounce_queue_lifetime 1d -# No sender rewrite: mail goes out as the natural user@. -for k in sender_canonical_maps smtp_header_checks; do +# Sender rewrite only with MAIL_SENDER; else mail goes out as the natural user@. +unset_keys="sender_canonical_maps smtp_header_checks" why="no sender rewrite" +if [[ -n "$EXPECT_SENDER" ]]; then + expect_pc sender_canonical_maps "static:$EXPECT_SENDER" + expect_pc sender_canonical_classes "envelope_sender, header_sender" + unset_keys="smtp_header_checks" why="no header rewrite" +fi +for k in $unset_keys; do v="$(pc "$k")" - [[ -z "$v" ]] && pass "$k unset (no sender rewrite)" || fail "$k = '$v', expected unset" + [[ -z "$v" ]] && pass "$k unset ($why)" || fail "$k = '$v', expected unset" done -v="$(postconf -n 2>/dev/null | grep -E '^(default|relay)_transport =')" -[[ -z "$v" ]] && pass "default_transport and relay_transport unset (outgoing mail allowed)" \ - || fail "outgoing mail blocked: ${v//$'\n'/; }" -check "root mail aliased (root: line in /etc/aliases)" grep -q '^root:' /etc/aliases -grep -qE "^root:[[:space:]]*${EXPECT_GMAIL_USER}[[:space:]]*$" /etc/aliases 2>/dev/null \ - && pass "root mail goes to $EXPECT_GMAIL_USER" || warn "root: alias in /etc/aliases is not $EXPECT_GMAIL_USER" +if [[ "$EXPECT_RELAY" == none ]]; then + skip "default_transport and relay_transport stay as Local only set them (MAIL_RELAY=none)" +else + v="$(postconf -n 2>/dev/null | grep -E '^(default|relay)_transport =')" + [[ -z "$v" ]] && pass "default_transport and relay_transport unset (outgoing mail allowed)" \ + || fail "outgoing mail blocked: ${v//$'\n'/; }" +fi +if [[ -n "$EXPECT_ROOT_TO" ]]; then + check "root mail aliased (root: line in /etc/aliases)" grep -q '^root:' /etc/aliases + sed -n 's/^root:[[:space:]]*\(.*[^[:space:]]\)[[:space:]]*$/\1/p' /etc/aliases 2>/dev/null | grep -qxF -- "$EXPECT_ROOT_TO" \ + && pass "root mail goes to $EXPECT_ROOT_TO" || warn "root: alias in /etc/aliases is not $EXPECT_ROOT_TO" +else + skip "root mail alias (no MAIL_ROOT_TO)" +fi check 'no cron mail from /etc/crontab (MAILTO="")' grep -qx 'MAILTO=""' /etc/crontab sp=/etc/postfix/sasl/sasl_passwd -if [[ -f "$sp" ]]; then - grep -q "^\[smtp.gmail.com\]:587 ${EXPECT_GMAIL_USER}:." "$sp" && pass "Gmail login set for $EXPECT_GMAIL_USER (password not shown)" \ - || fail "sasl_passwd has no login for $EXPECT_GMAIL_USER" - [[ "$(mode_of "$sp")" == 600 ]] && pass "sasl_passwd is 600" || fail "sasl_passwd mode is $(mode_of "$sp"), expected 600" - ls "$sp".* &>/dev/null && pass "sasl_passwd map built ($(basename "$(ls "$sp".* | head -1)"))" || fail "sasl_passwd map not built (postmap)" -else - fail "$sp missing" +if (( RELAY_LOGIN )); then + if [[ -f "$sp" ]]; then + # The line must start with "[host]:port user:" and have a password after it. + awk -v p="[$EXPECT_RELAY_HOST]:$EXPECT_RELAY_PORT $EXPECT_RELAY_USER:" \ + 'index($0, p) == 1 && length($0) > length(p) { f = 1 } END { exit !f }' "$sp" \ + && pass "$LOGIN_NAME set for $EXPECT_RELAY_USER (password not shown)" \ + || fail "sasl_passwd has no login for $EXPECT_RELAY_USER" + [[ "$(mode_of "$sp")" == 600 ]] && pass "sasl_passwd is 600" || fail "sasl_passwd mode is $(mode_of "$sp"), expected 600" + ls "$sp".* &>/dev/null && pass "sasl_passwd map built ($(basename "$(ls "$sp".* | head -1)"))" || fail "sasl_passwd map not built (postmap)" + else + fail "$sp missing" + fi fi -if [[ -n "$TEST_MAIL" ]]; then +if [[ "$EXPECT_RELAY" == none ]]; then + if [[ -n "$TEST_MAIL" ]]; then + warn "test email not sent to $TEST_MAIL: MAIL_RELAY=none keeps mail on this clone" + else + skip "test email (MAIL_RELAY=none keeps mail on this clone)" + fi +elif [[ -n "$TEST_MAIL" ]]; then echo "Template check from $(hostname) at $(date)" | mail -s "Template check: $(hostname)" "$TEST_MAIL" \ && pass "test email queued to $TEST_MAIL: check your inbox (and: journalctl -u postfix)" \ || fail "could not queue the test email" @@ -298,8 +371,14 @@ check "unattended-upgrades service enabled" systemctl is-enabled --quiet unatten [[ "$(uu_conf APT::Periodic::AutocleanInterval)" == 7 ]] && pass "autoclean every 7 days" || fail "AutocleanInterval is not 7" r="$(uu_conf Unattended-Upgrade::Automatic-Reboot)" [[ "$r" != true ]] && pass "no automatic reboot" || fail "automatic reboot is on (should be off)" -[[ "$(uu_conf Unattended-Upgrade::Mail)" == "$EXPECT_UU_MAIL" ]] && pass "upgrade reports mailed to $EXPECT_UU_MAIL" || fail "Unattended-Upgrade::Mail is not $EXPECT_UU_MAIL" -[[ "$(uu_conf Unattended-Upgrade::MailReport)" == on-change ]] && pass "MailReport on-change" || fail "MailReport is not on-change" +if [[ -n "$EXPECT_UU_MAIL" ]]; then + [[ "$(uu_conf Unattended-Upgrade::Mail)" == "$EXPECT_UU_MAIL" ]] && pass "upgrade reports mailed to $EXPECT_UU_MAIL" || fail "Unattended-Upgrade::Mail is not $EXPECT_UU_MAIL" + [[ "$(uu_conf Unattended-Upgrade::MailReport)" == on-change ]] && pass "MailReport on-change" || fail "MailReport is not on-change" +else + # UU_MAIL blank: the builder writes neither Mail nor MailReport. + v="$(uu_conf Unattended-Upgrade::Mail)" + [[ -z "$v" ]] && pass "no upgrade report mail (UU_MAIL blank)" || fail "Unattended-Upgrade::Mail is '$v', expected unset (UU_MAIL blank)" +fi [[ "$(uu_conf Unattended-Upgrade::Remove-Unused-Dependencies)" == true ]] && pass "Remove-Unused-Dependencies true" || fail "Remove-Unused-Dependencies not true" origins="$(apt-config dump 2>/dev/null)" missing=() @@ -326,26 +405,42 @@ fi if [[ "$LOGIN_USER" == root ]]; then warn "login user is root (Cloud-Init tab): shell, sudo and dotfile checks skipped; set a normal user there to test them" elif [[ -n "$LOGIN_USER" ]]; then - [[ "$(getent passwd "$LOGIN_USER" | cut -d: -f7)" == "$EXPECT_SHELL" ]] \ - && pass "login user $LOGIN_USER uses $EXPECT_SHELL" || fail "login user $LOGIN_USER does not use $EXPECT_SHELL" + if [[ -n "$EXPECT_SHELL" ]]; then + [[ "$(getent passwd "$LOGIN_USER" | cut -d: -f7)" == "$EXPECT_SHELL" ]] \ + && pass "login user $LOGIN_USER uses $EXPECT_SHELL" || fail "login user $LOGIN_USER does not use $EXPECT_SHELL" + else + skip "login user's shell (LOGIN_SHELL blank)" + fi id -nG "$LOGIN_USER" | grep -qw sudo && pass "$LOGIN_USER is in the sudo group" || fail "$LOGIN_USER is not in the sudo group" [[ -s "/home/$LOGIN_USER/.zshrc" ]] && pass "$LOGIN_USER got the dotfiles from /etc/skel" || warn "$LOGIN_USER has no ~/.zshrc" fi -grep -q "^DSHELL=$EXPECT_SHELL" /etc/adduser.conf && pass "adduser default shell $EXPECT_SHELL" || fail "adduser default shell is not $EXPECT_SHELL" -grep -q "^SHELL=$EXPECT_SHELL" /etc/default/useradd && pass "useradd default shell $EXPECT_SHELL" || fail "useradd default shell is not $EXPECT_SHELL" +if [[ -n "$EXPECT_SHELL" ]]; then + grep -q "^DSHELL=$EXPECT_SHELL" /etc/adduser.conf && pass "adduser default shell $EXPECT_SHELL" || fail "adduser default shell is not $EXPECT_SHELL" + grep -q "^SHELL=$EXPECT_SHELL" /etc/default/useradd && pass "useradd default shell $EXPECT_SHELL" || fail "useradd default shell is not $EXPECT_SHELL" +else + skip "adduser and useradd default shell (LOGIN_SHELL blank: OS default)" +fi # ============================================================================= section "needrestart, adverts, journal" -grep -q "\$nrconf{restart} = '$EXPECT_NEEDRESTART';" /etc/needrestart/conf.d/50-restart-mode.conf 2>/dev/null \ - && pass "needrestart restart mode '$EXPECT_NEEDRESTART'" || fail "needrestart restart mode not '$EXPECT_NEEDRESTART'" +if [[ -n "$EXPECT_NEEDRESTART" ]]; then + grep -q "\$nrconf{restart} = '$EXPECT_NEEDRESTART';" /etc/needrestart/conf.d/50-restart-mode.conf 2>/dev/null \ + && pass "needrestart restart mode '$EXPECT_NEEDRESTART'" || fail "needrestart restart mode not '$EXPECT_NEEDRESTART'" +else + skip "needrestart restart mode (NEEDRESTART_MODE blank: OS default)" +fi [[ ! -e /etc/apt/apt.conf.d/20apt-esm-hook.conf ]] && pass "apt Pro/ESM hook disabled" || fail "apt Pro/ESM hook still active" grep -q '^ENABLED=0' /etc/default/motd-news 2>/dev/null && pass "motd news off" || fail "motd news still on" [[ ! -x /etc/update-motd.d/91-contract-ua-esm-status ]] && pass "Pro status removed from the login message" || fail "Pro status still in the login message" -cap="$(sed -n 's/^SystemMaxUse=\([0-9]*\)M$/\1/p' /etc/systemd/journald.conf.d/50-size.conf 2>/dev/null)" -fs_mb=$(( $(df --output=size -k /var/log | tail -1) / 1024 )) -want_cap=$(( fs_mb * EXPECT_JOURNAL_PERCENT / 100 )); (( want_cap > EXPECT_JOURNAL_MAX_MB )) && want_cap=$EXPECT_JOURNAL_MAX_MB -[[ "$cap" == "$want_cap" ]] && pass "journal capped at ${cap}M (min ${EXPECT_JOURNAL_MAX_MB}M, ${EXPECT_JOURNAL_PERCENT}% of ${fs_mb}M)" \ - || fail "journal cap is '${cap:-unset}', expected ${want_cap}M" +if [[ -n "$EXPECT_JOURNAL_MAX_MB" ]]; then + cap="$(sed -n 's/^SystemMaxUse=\([0-9]*\)M$/\1/p' /etc/systemd/journald.conf.d/50-size.conf 2>/dev/null)" + fs_mb=$(( $(df --output=size -k /var/log | tail -1) / 1024 )) + want_cap=$(( fs_mb * EXPECT_JOURNAL_PERCENT / 100 )); (( want_cap > EXPECT_JOURNAL_MAX_MB )) && want_cap=$EXPECT_JOURNAL_MAX_MB + [[ "$cap" == "$want_cap" ]] && pass "journal capped at ${cap}M (min ${EXPECT_JOURNAL_MAX_MB}M, ${EXPECT_JOURNAL_PERCENT}% of ${fs_mb}M)" \ + || fail "journal cap is '${cap:-unset}', expected ${want_cap}M" +else + skip "journal cap (JOURNAL_MAX_MB blank: OS default)" +fi echo " journal now uses: $(journalctl --disk-usage 2>/dev/null | grep -o '[0-9.]*[KMG]' | head -1)" # =============================================================================