Hi,
The clickhouse-operator chart's serviceMonitor.enabled creates a ServiceMonitor with two endpoints: ch-metrics and op-metrics. Neither covers ClickHouse Keeper.
Once Keeper's own native Prometheus metrics are enabled (via the clickhouse chart), there's no way to scrape it through this operator. A ServiceMonitor/Prom scrape job must be separately/manually created. Unlike CH server metrics, Keeper doesn't use the metrics-exporter's SQL-aggregation approach. It exposes standard Prometheus metrics directly on its own pods. So this isn't a matter of extending the exporter, it's simply a missing scrape target.
I suggest adding a third keeper-metrics endpoint to the existing templates/servicemonitor.yaml, rolled up under serviceMonitor.enabled like the other two.
The operator already labels every Keeper Service it manages: clickhouse-keeper.altinity.com/app: chop, and clickhouse's chk.yaml always names the metrics port metrics regardless of its numeric value, so matchLabels: {clickhouse-keeper.altinity.com/app: chop} + port: metrics picks up every Keeper instance the operator watches, no user config required.
Anyone wanting just one CHK's Keeper can filter in PromQL labels, similar to CH (which uses chi= or /exported_namespace= labels).
Clusters without Keeper deployed (keeper.enabled: false) are unaffected. The selector will match nothing, same as any ServiceMonitor without matching targets.
While I suggest simply using the single serviceMonitor.enabled flag. Alternatively, could have an independently-toggleable serviceMonitor.keeperMetrics.enabled (default false), decoupled from the parent flag, for anyone who wants CH server + operator metrics but not Keeper's. But you might consider this to be an edge-case, and not worth the extra effort/complexity.
Cheers!
Hi,
The clickhouse-operator chart's
serviceMonitor.enabledcreates a ServiceMonitor with two endpoints: ch-metrics and op-metrics. Neither covers ClickHouse Keeper.Once Keeper's own native Prometheus metrics are enabled (via the clickhouse chart), there's no way to scrape it through this operator. A ServiceMonitor/Prom scrape job must be separately/manually created. Unlike CH server metrics, Keeper doesn't use the metrics-exporter's SQL-aggregation approach. It exposes standard Prometheus metrics directly on its own pods. So this isn't a matter of extending the exporter, it's simply a missing scrape target.
I suggest adding a third keeper-metrics endpoint to the existing
templates/servicemonitor.yaml, rolled up underserviceMonitor.enabledlike the other two.The operator already labels every Keeper Service it manages:
clickhouse-keeper.altinity.com/app: chop, and clickhouse's chk.yaml always names the metrics portmetricsregardless of its numeric value, so matchLabels: {clickhouse-keeper.altinity.com/app: chop} + port: metrics picks up every Keeper instance the operator watches, no user config required.Anyone wanting just one CHK's Keeper can filter in PromQL labels, similar to CH (which uses
chi=or/exported_namespace=labels).Clusters without Keeper deployed (
keeper.enabled: false) are unaffected. The selector will match nothing, same as any ServiceMonitor without matching targets.While I suggest simply using the single
serviceMonitor.enabledflag. Alternatively, could have an independently-toggleableserviceMonitor.keeperMetrics.enabled(default false), decoupled from the parent flag, for anyone who wants CH server + operator metrics but not Keeper's. But you might consider this to be an edge-case, and not worth the extra effort/complexity.Cheers!