fix(roles/cloud_init): reload NetworkManager after removing 99-cloud-init.conf

CHANGELOG.md

@@ -147,6 +147,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+* **role:cloud_init**: NetworkManager manages `/etc/resolv.conf` right after cloud-init's `99-cloud-init.conf` is removed, instead of only after the next reboot or NetworkManager restart.
 * **role:nextcloud**: A Nextcloud upgrade no longer logs an error from a repair step of the `dav` app and no longer rebuilds the calendar search index every time.
 * **role:nextcloud**: The admin section of the `support` app is hidden again.
 * **role:monitoring_plugins**: The package install works on SLE, where it failed with `No provider of 'linuxfabrik-monitoring-plugins-<version>*' found`.

extensions/molecule/setup_basic/prepare.yml

@@ -35,3 +35,53 @@
       changed_when: true
       when:
         - '__molecule__grubenv_stat_result["stat"]["islnk"] | d(false) | bool'
+
+
+# Put back what cloud-init's sysconfig renderer leaves on a provider image that hands out DNS
+# servers (Rocky 8 and 9 on Hetzner, for example): "dns = none" in 99-cloud-init.conf, already
+# loaded by NetworkManager. The test images do not get DNS servers from cloud-init, so without
+# this the cloud_init role would have nothing to clean up and verify.yml would pass on a host that
+# was never in that state.
+- name: 'Leave the NetworkManager config of cloud-init behind'
+  hosts: 'systems_under_test'
+
+  tasks:
+
+    - name: 'Gather the service facts to see whether NetworkManager runs'
+      ansible.builtin.service_facts: # yamllint disable-line rule:empty-values
+
+    - name: 'Create /etc/NetworkManager/conf.d/99-cloud-init.conf'
+      ansible.builtin.copy:
+        content: |
+          [main]
+          dns = none
+        dest: '/etc/NetworkManager/conf.d/99-cloud-init.conf'
+        owner: 'root'
+        group: 'root'
+        mode: 0o644
+      when:
+        - 'ansible_facts["services"]["NetworkManager.service"]["state"] | d("") == "running"'
+
+    - name: 'systemctl reload NetworkManager.service'
+      ansible.builtin.service:
+        name: 'NetworkManager'
+        state: 'reloaded'
+      when:
+        - 'ansible_facts["services"]["NetworkManager.service"]["state"] | d("") == "running"'
+
+    - name: 'busctl get-property ... DnsManager Mode'
+      ansible.builtin.command: >-
+        busctl get-property org.freedesktop.NetworkManager
+        /org/freedesktop/NetworkManager/DnsManager org.freedesktop.NetworkManager.DnsManager Mode
+      changed_when: false
+      register: '__molecule__nm_dns_mode_result'
+      when:
+        - 'ansible_facts["services"]["NetworkManager.service"]["state"] | d("") == "running"'
+
+    - name: 'Assert that NetworkManager no longer manages /etc/resolv.conf'
+      ansible.builtin.assert:
+        that:
+          - '__molecule__nm_dns_mode_result["stdout"] == "s \"none\""'
+        fail_msg: '{{ __molecule__nm_dns_mode_result["stdout"] }}'
+      when:
+        - 'ansible_facts["services"]["NetworkManager.service"]["state"] | d("") == "running"'

extensions/molecule/setup_basic/verify.yml

@@ -2,6 +2,30 @@
   hosts: 'systems_under_test'
   gather_facts: false
   tasks:
+    # prepare.yml left cloud-init's "dns = none" behind on every host that runs NetworkManager. Once
+    # the cloud_init role removed it, NetworkManager has to be managing /etc/resolv.conf again
+    # without a restart in between. The DNS mode tells, the header of /etc/resolv.conf does not:
+    # NetworkManager leaves the file it wrote before untouched in mode "none".
+    - name: 'Gather the service facts to see whether NetworkManager runs'
+      ansible.builtin.service_facts: # yamllint disable-line rule:empty-values
+
+    - name: 'busctl get-property ... DnsManager Mode'
+      ansible.builtin.command: >-
+        busctl get-property org.freedesktop.NetworkManager
+        /org/freedesktop/NetworkManager/DnsManager org.freedesktop.NetworkManager.DnsManager Mode
+      changed_when: false
+      register: '__molecule__nm_dns_mode_result'
+      when:
+        - 'ansible_facts["services"]["NetworkManager.service"]["state"] | d("") == "running"'
+
+    - name: 'Assert that NetworkManager manages /etc/resolv.conf again'
+      ansible.builtin.assert:
+        that:
+          - '__molecule__nm_dns_mode_result["stdout"] != "s \"none\""'
+        fail_msg: '{{ __molecule__nm_dns_mode_result["stdout"] }}'
+      when:
+        - 'ansible_facts["services"]["NetworkManager.service"]["state"] | d("") == "running"'
+
     # Running a plugin instead of stat-ing it proves the deployed payload executes and its bundled
     # runtime resolves, which a stat cannot tell you.
     - name: '/usr/lib64/nagios/plugins/about-me'

roles/cloud_init/README.md

@@ -5,7 +5,7 @@ This role removes the `cloud-init` package and cleans up artifacts that the pack
 Concretely, the role:
 
 * Removes the `cloud-init` package via the OS package manager.
-* Removes `/etc/NetworkManager/conf.d/99-cloud-init.conf` so NetworkManager stops managing `/etc/resolv.conf` on its behalf.
+* Removes `/etc/NetworkManager/conf.d/99-cloud-init.conf` and reloads NetworkManager if it is running. cloud-init writes `dns = none` there, so NetworkManager does not manage `/etc/resolv.conf`. After the reload it writes the DNS servers of the active connection into `/etc/resolv.conf`.
 * Removes `/etc/cloud/cloud.cfg.rpmsave` left over from the package removal.
 
 Depending on the cloud provider, `cloud-init` changes SSH security settings, which we do not want. Note that removing `cloud-init` may break first-boot integrations of the cloud provider (metadata, SSH key injection, hostname provisioning, ...). Run this role only on hosts that have already been provisioned, or on installations where you do not rely on cloud-init at all.
@@ -19,7 +19,7 @@ Depending on the cloud provider, `cloud-init` changes SSH security settings, whi
 `cloud_init`
 
 * Removes the cloud-init package and its leftover config files.
-* Triggers: none.
+* Triggers: NetworkManager.service reload.
 
 
 ## License

roles/cloud_init/handlers/main.yml

@@ -0,0 +1,11 @@
+# NetworkManager is not owned by this role. A reload re-reads conf.d and re-evaluates the DNS mode
+# without touching the connections (NetworkManager 1.56, src/core/dns/nm-dns-manager.c,
+# config_changed_cb), so lfops__skip_restart_handlers does not apply.
+# ansible.builtin.service with state 'reloaded' would start a stopped NetworkManager, while
+# try-reload-or-restart leaves it alone. NetworkManager has an ExecReload, so this never restarts.
+# rc 5 means the unit does not exist; a host without NetworkManager has nothing to reload.
+- name: 'cloud_init: systemctl try-reload-or-restart NetworkManager.service'
+  ansible.builtin.command: 'systemctl try-reload-or-restart NetworkManager.service' # noqa command-instead-of-module
+  register: '__cloud_init__networkmanager_reload_result'
+  changed_when: '__cloud_init__networkmanager_reload_result["rc"] == 0'
+  failed_when: '__cloud_init__networkmanager_reload_result["rc"] not in [0, 5]'

roles/cloud_init/tasks/main.yml

@@ -5,10 +5,13 @@
       name: 'cloud-init'
       state: 'absent'
 
-  - name: 'rm /etc/NetworkManager/conf.d/99-cloud-init.conf # this prevents NetworkManager from managing /etc/resolv.conf'
+  # cloud-init's sysconfig renderer writes "dns = none" there, which keeps NetworkManager from
+  # managing /etc/resolv.conf. NetworkManager only notices the removal after a reload.
+  - name: 'rm /etc/NetworkManager/conf.d/99-cloud-init.conf'
     ansible.builtin.file:
       path: '/etc/NetworkManager/conf.d/99-cloud-init.conf'
       state: 'absent'
+    notify: 'cloud_init: systemctl try-reload-or-restart NetworkManager.service'
 
   - name: 'rm /etc/cloud/cloud.cfg.rpmsave'
     ansible.builtin.file: