plugin-metrics

OpenEverest plugin that charts database instance metrics right on the instance page, so users don't have to switch to Grafana for a quick look.

It adds a Metrics tab with a small dashboard (CPU, memory and engine-specific panels) and a Last hour / Last 24 hours switch. A search box adds any other metric the instance exposes as an extra chart for the session — nothing is saved; for deeper digging use Grafana, PMM or similar.

How it works

flowchart LR
  UI["Instance page<br/>Metrics tab"] -->|user token| ES[everest-server<br/>plugin proxy]
  ES --> BE[plugin backend]
  BE -->|1. read the Instance as the user| API[Everest API]
  BE -->|2. pick a source that monitors it| SRC{{Sources}}
  SRC -->|3. scoped query| P[(Prometheus)]
  SRC -->|3. scoped query| PMM[(PMM)]

Loading

1. Every request is authorized by reading the Instance from the Everest API with the caller's token, so users only see metrics of instances they can read.
2. The backend asks each configured source whether it monitors the instance and uses the first one that does, trying the sources the provider's dashboard has queries for first.
3. It runs the dashboard's queries for that source. Queries come from the plugin's configuration, never from the browser, and are always scoped to the instance, because neither Prometheus nor a shared PMM server isolates tenants.

Sources

A source is one monitoring system (backend/internal/source/source.go):

type Source interface {
    Type() string // keys this source's queries in dashboards, e.g. "prometheus"
    Status(ctx, target) (Status, error)
    QueryRange(ctx, target, query, window) ([]Series, error)
}

Source	Monitored when	Scoping
prometheus	A PodMonitor or ServiceMonitor for the instance exists (see below)	${selector} expands to the instance's namespace and scrape jobs
pmm	The instance's monitoring component references a PMM MonitoringConfig	${selector} expands to the instance's namespace and core_openeverest_io_instance labels, also enforced as extra_filters[]

The Prometheus source finds an instance's monitors in its namespace by either label set:

• app.kubernetes.io/managed-by=everest + app.kubernetes.io/instance=<name> for monitors the provider creates itself;
• core.openeverest.io/instance=<name> for monitors a database operator creates (the operator owns managed-by), which the provider labels through the operator's API, e.g. MariaDB spec.inheritMetadata or k8ssandra telemetry.prometheus.commonLabels.

A PodMonitor's scrape job is <namespace>/<name>. A ServiceMonitor's jobs are the Services it selects (or their jobLabel value), so the plugin also lists Services.

The PMM source reads the instance's MonitoringConfig (spec.components.monitoring.parameters.monitoringConfigName) and the Secret it references, and queries that PMM server's Prometheus API (<url>/prometheus/api/v1) with the Secret's apiKey as a bearer token, honoring verifyTLS. Providers label every series an instance pushes to PMM with namespace and core_openeverest_io_instance (Percona providers do so through pmm-admin --custom-labels), which keeps queries to one instance on a PMM server that monitors many. Series are named after the database pod (PMM's service_name, <namespace>-<pod>, without the namespace). The labels do not identify the Kubernetes cluster yet, so a PMM server shared by several clusters should not hold instances with the same namespace and name.

Adding another system means adding a Source and its queries to the panels that support it. The frontend renders whatever series it gets back and doesn't change.

Pod CPU and memory

Every dashboard starts with CPU usage and Memory usage of the instance's pods, whichever source serves the engine's panels and even when the instance has no monitoring enabled. They come from the container metrics (cAdvisor) the cluster's Prometheus scrapes from every kubelet:

• sum by (pod) (rate(container_cpu_usage_seconds_total[...])) and sum by (pod) (container_memory_working_set_bytes), one line per pod.
• Queries are scoped to the instance's namespace and the pods labelled core.openeverest.io/instance=<name> now, so a pod replaced within the time range shows only under its current name.
• A dashed limit line shows the sum of each pod's container limits (containers without a limit add nothing): one line when all pods share it, otherwise one per core.openeverest.io/component, or per pod.
• The panels are left out when Prometheus is unreachable or has no container metrics for the pods; the rest of the dashboard is unaffected.

Dashboards

Panels are data, keyed by provider, in backend/internal/dashboard/dashboards.yaml:

dashboards:
  milvus:
    panels:
      - id: vectors
        title: Vectors inserted per second
        unit: ops              # cores, bytes, Bps, ops (per second), ms, or empty
        queries:
          prometheus: >-
            sum(rate(milvus_proxy_insert_vectors_count{${selector}}[${rate}]))

${rate} is a rate() window matched to the selected time range. Each series is named after the labels the query keeps, e.g. the pod. A panel may hold a query per source, e.g. prometheus: and pmm:, and is shown for whichever source monitors the instance.

Built-in dashboards:

Provider	Panels
milvus	Requests/s, search/query latency p99, vectors inserted/s, collections
valkey	Commands/s, keyspace hit ratio, connected clients, memory, keys and network traffic by shard
mariadb	Queries/s, slow queries/s, connections, running threads, InnoDB buffer pool hit ratio, network traffic
mssql	Batch requests/s, transactions/s, user connections, buffer cache hit ratio, server memory, database size
provider-cassandra	Client requests/s, latency p99 and timeouts/s by request type, native clients, live disk space, pending compactions
provider-cloudnative-pg	Transactions/s, rows written/s, connections, cache hit ratio, replication lag, database size
provider-seaweedfs	S3 requests/s, latency p99 by method, error responses/s by code, S3 traffic, bucket size and objects, volume server disk used, volume slots used
tidb	Queries/s by type, query latency p99, failed queries/s, connections, transactions/s, TiKV storage used
percona-server-mongodb (PMM)	Operations/s, average latency by operation type, connections, WiredTiger cache used, operations queued on locks, replication lag by replica set
provider-percona-postgresql (PMM)	Transactions/s, rows written/s, connections, cache hit ratio, replication lag, database size
percona-xtradb-cluster (PMM or Prometheus)	Queries/s, connections, Galera flow control paused, Galera receive queue, InnoDB buffer pool hit ratio, network traffic

Each dashboard expects the provider's usual exporter (Milvus built-in, redis_exporter, mysqld_exporter, sql_exporter with the mssql-operator collectors, k8ssandra's modern metrics endpoint, the CloudNativePG exporter, SeaweedFS built-in, TiDB/TiKV/PD built-in, or the PMM client's exporters) and a monitor or MonitoringConfig the plugin can discover (see Sources).

Custom dashboards

Admins can add dashboards for more providers or replace the built-in ones without rebuilding the plugin, through a ConfigMap in the same format. Set them in the chart values, which renders the ConfigMap:

dashboards:
  overrides:
    valkey:
      panels:
        - id: commands
          title: Commands per second
          unit: ops
          queries:
            prometheus: >-
              sum by (valkey_io_shard_index) (rate(redis_commands_processed_total{${selector}}[${rate}]))
        - id: memory
          title: Memory used by shard
          unit: bytes
          queries:
            prometheus: >-
              sum by (valkey_io_shard_index) (redis_memory_used_bytes{${selector}})
plugin:
  extensionPoints:
    - type: clusterDetailTab
      path: metrics
      label: Metrics
      providers: [milvus, valkey]   # show the tab for the new provider

or point dashboards.existingConfigMap at a ConfigMap you manage yourself (key dashboards.yaml, content dashboards: {...}), e.g. with GitOps.

• A provider's dashboard replaces its built-in one as a whole; {panels: []} hides it (the metric search stays available).
• Edits to the ConfigMap apply within about a minute, without a restart.
• Panel ids starting with pod- are reserved for the pod CPU and memory panels.
• Every query must use ${selector}. A file with an invalid dashboard or an unscoped query is rejected as a whole: the plugin keeps the last good dashboards and logs why.

Requirements

• OpenEverest with plugin support (plugins.extensions.openeverest.io CRD).
• The Prometheus Operator and a Prometheus that selects the instances' PodMonitors/ServiceMonitors (for kube-prometheus-stack, label them release: <release-name>, e.g. via the Milvus provider's podMonitorLabels).
• Prometheus monitoring enabled on the instance (for Milvus, the Monitoring → Prometheus toggle).
• For pod CPU and memory: a Prometheus at prometheus.url that scrapes the kubelets' cAdvisor endpoint, as kube-prometheus-stack does by default. The plugin lists pods in every namespace to scope the queries and read limits.
• For PMM: PMM 3 and a provider that labels the series it pushes to PMM with the instance (Percona providers for MongoDB, PostgreSQL and XtraDB Cluster), with monitoring enabled on the instance through a MonitoringConfig. The plugin reuses that MonitoringConfig's token, so its ServiceAccount can read MonitoringConfigs and Secrets in every namespace.

Installation

helm install plugin-metrics charts/plugin-metrics \
  --namespace everest-system \
  --set prometheus.url=http://kube-prometheus-stack-prometheus.monitoring.svc:9090

Value	Default	Description
prometheus.url	http://kube-prometheus-stack-prometheus.monitoring.svc:9090	In-cluster Prometheus HTTP API
dashboards.overrides	{}	Dashboards added or replaced by provider (see Custom dashboards)
dashboards.existingConfigMap	""	Your own ConfigMap with dashboards.yaml; wins over overrides
plugin.extensionPoints[0].providers	[milvus]	Providers that get the Metrics tab
everestAPIURL	discovered	Everest API the backend authorizes against

API

Served under /v1/clusters/{cluster}/plugins/plugin-metrics by everest-server.

Endpoint	Returns
GET /api/dashboard?namespace=&instance=	The active source and its status, and the panels it can fill
GET /api/panels/{id}?namespace=&instance=&range=1h|24h	The panel's series; limit lines have kind: limit
GET /api/metrics?namespace=&instance=	The instance's raw metrics with type and description (explorer sources only)
GET /api/explore?namespace=&instance=&metric=&type=&range=	One raw metric charted by type: counters as a per-second rate, histograms as p99, summaries as the average, gauges as is

Explored queries are built on the backend from a validated metric name and are scoped like dashboard queries, so they cannot read other instances' metrics.

Development

npm install
make test             # backend + frontend tests
make build-frontend   # dist/main.js
make docker-build IMG=<registry>/plugin-metrics:dev

Local environment (Tilt)

cp dev/.env.example dev/.env   # set OPENEVEREST_VERSION while v2 is in pre-release
make dev-up                    # k3d cluster + tilt up

dev/Tiltfile installs the released OpenEverest core, kube-prometheus-stack, the Milvus provider and a small standalone instance with Prometheus monitoring on (dev/demo-instance.yaml), then rebuilds and redeploys the plugin on every change. Open the UI at http://localhost:8080 (admin / UI_ADMIN_PASSWORD) and Prometheus at http://localhost:9090. Each dependency can be turned off in dev/.env to reuse one you already run. make dev-down stops Tilt; make dev-destroy also deletes the cluster.

Releasing

Push a vX.Y.Z (or vX.Y.Z-<suffix>) tag. The release workflow builds the bundle and a multi-arch image, pushes the image and the Helm chart to GHCR (oci://ghcr.io/openeverest/charts/plugin-metrics) and creates a GitHub release with notes.

The frontend is an ES module loaded by the Everest UI and shares the host's React (see vite.config.ts). The backend embeds it and serves it at /main.js.

Roadmap

• More sources: Prometheus endpoints registered through MonitoringConfig.
• ServiceMonitor discovery and dashboards for more providers.