Prometheus metrics for MongoDB instances created via MongoDB Community Operator

Alexandru_Marian_Lita · 2024-04-09T01:40:41.759Z

I’m having trouble monitoring MongoDB instances created via the MongoDB Community Operator with Prometheus Scrape Jobs. The metrics endpoint is always protected, and I can’t disable it. Additionally, I can’t add the annotations needed for service discovery on the Kubernetes service resource created by the operator because only the StatefulSet resource is configurable. I’m considering using a MongoDB exporter, but I’m unsure if it can handle metrics from multiple instances. Any advice or configuration examples would be appreciated.

Dan_Mckean · 2024-04-09T13:34:39.176Z

Hi, we have a docs page in the repo about this:

github.com

mongodb/mongodb-kubernetes-operator/blob/master/docs/prometheus/README.md

# Use Prometheus with your MongoDB Resource

 You can use the [mongodb-prometheus-sample.yaml](mongodb-prometheus-sample.yaml) file to 
 deploy a MongoDB resource in your Kubernetes cluster, with a
[`ServiceMonitor`](https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/user-guides/getting-started.md#related-resources)
to indicate to Prometheus how to consume metrics data from 
it.

The sample specifies a simple MongoDB resource with one user,
and the `spec.Prometheus` attribute with basic HTTP 
authentication and no TLS. The sample lets you test
the metrics that MongoDB sends to Prometheus.

## Quick Start

We tested this setup with version 0.54 of the [Prometheus
Operator](https://github.com/prometheus-operator/prometheus-operator).

### Prerequisites

This file has been truncated. show original

But it covers using a service monitor, instead of service discovery.

We don’t allow configuring prometheus without any password.

Does this offer what you need?

Alexandru_Marian_Lita · 2024-04-12T16:17:53.215Z

Hi,

Opentelemtry does not fully support service monitors, and I had to switch to a different approach to scrape the service via a static target.

Now I’m trying to build a Grafana Dashboard from scratch, because the old one does not seem to work anymore on the latest Grafana version and the data returned by the queries does not seem correct.

For example, “mongodb_opLatencies_reads_ops” value grows over time, I don’t see any moment where the value decreases. I currently don’t execute anything to the database, so I don’t understand why this grows and grows without stop.

Is there any documentation or new dashboard that I can consume with the latest version of the operator? I’m trying to monitor the MongoDB instances via the default metrics.

Thank you

Dan_Mckean · 2024-04-15T12:44:56.071Z

Hi, I chatted to the Engineering team.

Unfortunately we’d need to spend some time investigating and recreating this to be able to provide a meaningful reply.

I’ve created a ticket on our internal backlog to pick this up and see what we can offer!

I’m not sure right now when that’s likely to happen but when it does we’ll reply back here.

Alexandru_Marian_Lita · 2024-04-19T12:42:12.162Z

Hi,

Thank you for your prompt reply, I’ll wait for a reply on this thread.