Repository navigation
kolla: Scrape the Ceph mgr when OSISM deploys Ceph - #330
Merged
Merged
Conversation
The prometheus module of the Ceph mgr is enabled on every cephadm deployment (ceph_mgr_modules), but the Prometheus that OSISM deploys does not scrape it. On 2024.1 and 2024.2 the scrape job came from kolla-operations (prometheus.yml.d/50-ceph.yml). kolla-operations is not used from 2025.1 on, so since then a deployment has no Ceph metrics in Prometheus unless the operator adds a prometheus.yml.d overlay. kolla-ansible already has a ceph_mgr_exporter job for this, gated on enable_prometheus_ceph_mgr_exporter and fed by prometheus_ceph_mgr_exporter_endpoints. Turn it on when Prometheus is enabled, kolla-operations is not, and the ceph-mgr group has hosts, which is the case when OSISM deploys Ceph; with an external Ceph the group is empty. The endpoints are all ceph-mgr hosts on port 9283, at the same address kolla uses for the node exporter on those hosts (the API address, or prometheus_target_address). The module listens on all addresses. Standby mgrs answer with an empty body, so listing all of them makes the metrics follow the active mgr without duplicates. Tested on a 2026.1 deployment with three mgrs and Prometheus on the manager: osism apply prometheus renders the job with the three API addresses, all targets are up, ceph_health_status exists once, and after ceph mgr fail it comes from the new active mgr. The enable expression evaluates to false without a ceph-mgr group, with Prometheus off, and with kolla-operations on. A deployment that already scrapes the mgr through its own overlay gets the metrics twice, under two job names, until the overlay is removed. UpgradeImpact Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Roger Luethi <luethi@osism.tech>
ideaship
marked this pull request as ready for review
October 16, 2026 16:17
berendt
pushed a commit
to osism/testbed
that referenced
this pull request
Oct 16, 2026
prometheus.yml.d/50-ceph.yml adds a "ceph" scrape job for the Ceph mgr prometheus module, with one target per host in the mons group at its storage_interface address. osism/defaults now enables kolla-ansible's own ceph_mgr_exporter job whenever OSISM deploys Ceph and Prometheus is on, with every ceph-mgr host as a target (osism/defaults#330). With both in place, Prometheus scrapes every Ceph series twice under two job names. Remove the overlay and its ansible-lint exclusion, which existed only because the file is a Jinja template. This depends on osism/defaults#330. Until a release carries it, deployments of that release lose the Ceph scrape. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Roger Luethi <luethi@osism.tech>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Ceph mgr prometheus module is enabled on every cephadm deployment, but the Prometheus that OSISM deploys does not scrape it. On 2024.1 and 2024.2 the scrape job came from kolla-operations (
prometheus.yml.d/50-ceph.yml). kolla-operations is not used from 2025.1 on, so since then a deployment has no Ceph metrics unless the operator adds aprometheus.yml.doverlay.kolla-ansible already has a
ceph_mgr_exporterscrape job. This turns it on inall/099-kolla.yml:enable_prometheus_ceph_mgr_exporteris true when Prometheus is enabled, kolla-operations is not, and theceph-mgrgroup has hosts, i.e. when OSISM deploys Ceph. With an external Ceph the group is empty.prometheus_ceph_mgr_exporter_endpointslists every ceph-mgr host on port 9283, at the address kolla uses for the node exporter on that host (the API address, orprometheus_target_address). Standby mgrs answer with an empty body, so the metrics follow the active mgr without duplicates.Tested on a 2026.1 deployment with three mgrs: the job renders with the three API addresses, all targets are up,
ceph_health_statusexists once, and afterceph mgr failit comes from the new active mgr. The enable expression is false without aceph-mgrgroup, with Prometheus off, and with kolla-operations on.Upgrade impact: a deployment that already scrapes the mgr through its own overlay gets every Ceph series twice, under two job names, until it removes the overlay. testbed has such an overlay:
🤖 Generated with Claude Code