Understanding Prometheus Metrics

Prometheus is an open-source monitoring system designed to collect, store, and query real-time metrics from systems and applications. Metrics are fundamental to system and application monitoring, and Prometheus defines a specific format for metric data.

Metric Format: Prometheus metric data consists of a metric name, a set of key-value pairs called labels, a timestamp, and a numeric value. The metric name and labels together identify a specific metric, and the numeric value represents the measurement of that metric at the given timestamp. Here is an example of a metric in Prometheus format:

http_requests_total{method="POST", handler="/api/users", status="200"} 2436

In this example, http_requests_total is the metric name, and method, handler, and status are labels with their corresponding values. The value 2436 represents the number of HTTP requests made to the /api/users handler using the POST method that resulted in a 200 status code.

In the UI of prometheus you can query and see the metrics and his values

Metric Types

Prometheus defines four types of metrics: counters, gauges, histograms, and summaries.

• Counters: Counters represent a cumulative metric that monotonically increases over time. They are often used to track the number of events, such as requests or errors.
• Gauges: Gauges represent a metric that can go up or down over time. They are often used to track values that fluctuate, such as memory usage or CPU load.
• Histograms: Histograms represent a metric that is divided into buckets based on a set of predefined values. They are often used to track the distribution of values, such as response times or request sizes.
• Summaries: Summaries represent a metric that calculates the total count of events and their sum. They are often used to track the distribution of values, such as request durations or response sizes.

Metric Attributes

Prometheus also supports metric attributes, which provide additional information about metrics. Metric attributes are metadata that describe a metric and are not part of the metric value itself. For example, an attribute might describe the units of a metric or the source of the metric data.

Prometheus defines two types of metric attributes:

• Help Text: Help text provides a description of the metric and its intended use. It is used to help users understand the purpose and context of a metric. Help text is defined using a HELP statement in the Prometheus metric format.
• Type: Type describes the type of the metric, such as counter, gauge, histogram, or summary. It is used to help users understand the behavior and interpretation of the metric. Type is defined using a TYPE statement in the Prometheus metric format.

Here is an example of a metric with help text and type attributes:

# HELP http_requests_total The total number of HTTP requests.
# TYPE http_requests_total counter
http_requests_total{method="POST", handler="/api/users", status="200"} 2436

In this example, the HELP statement provides a description of the metric, and the TYPE statement identifies it as a counter.

Metric Scraping

Prometheus scrapes metrics from targets, such as application servers, using a pull model. It sends HTTP requests to each target, which returns the metrics data in the Prometheus format. Prometheus then stores the metric data in a time-series database and exposes an API for querying the data.

Conclusion

Prometheus provides a powerful and flexible system for collecting, storing, and querying metric data. By understanding the metric format and the different types of metrics, you can effectively monitor your systems and applications and gain valuable insights into their performance and behavior.