1.4.3. Performance and Usage Diagnosis enabler

1.4.3.1. Introduction

Performance and Usage Diagnosis (PUD) enabler aims at collecting performance metrics from monitored targets by scraping metrics HTTP endpoints on them and highlighting potential problems in the ASSIST-IoT platform, so that it could notify platform administrator to fine tuning architecture’s resources. For this purpose we use Prometheus, an open-source software that collects metrics from targets by “scraping” metrics HTTP endpoints. Supported “targets” include kube-state-metrics for monitoring every kubernetes cluster used in the project, node-exporter metrics for monitoring hardware, OS metrics exposed by *NIX kernels, as well as other important metrics for the rest of the enablers used in the architecture. Below is a high-level architecture of Performance and Usage Diagnosis enabler.

PUD Enabler reaches at the end of the ASSIST-IoT project TRL6.

1.4.3.2. Features

1.4.3.2.1. Performance and Usage Diagnosis (PUD) enabler’s features

• Prometheus Server is an open-source monitoring framework. It provides out-of-the-box monitoring capabilities for the Kubernetes container orchestration platform. Its main features are:

  1. Metric Collection: Prometheus uses the pull model to retrieve metrics over HTTP. There is an option to push metrics to Prometheus using Pushgateway for use cases where Prometheus cannot Scrape the metrics.

  2. Metric Endpoint: The systems that you want to monitor using Prometheus should expose the metrics on an /metrics endpoint. Prometheus uses this endpoint to pull the metrics in regular intervals.

  3. PromQL: Prometheus comes with PromQL, a very flexible query language that can be used to query the metrics in the Prometheus dashboard. Also, the PromQL query will be used by Prometheus UI and Grafana to visualize metrics.

  4. Prometheus Exporters: Exporters are libraries which converts existing metric from third-party apps to Prometheus metrics format. There are many official and community Prometheus exporters. One example is, Kube State metrics, a service which talks to Kubernetes API server to get all the details about all the API objects like deployments, pods, daemonsets etc.

  5. TSDB (time-series database): Prometheus uses TSDB for storing all the data. By default, all the data gets stored locally. However, there are options to integrate remote storage for Prometheus TSDB.

• Prometheus-es-adapter is a read and write adapter for integrading LTSE’s elastic search as prometheus’ persistent storage.

• Prom-target-api is an API for dynamically add, update and delete monitored targets.

• Grafana is a multi-platform open source analytics and interactive visualization web application. It’s used for creating and visualizing dashboads with graphs generated by prometheus metrics for more user friendly monitoring experience.

• Kube-state-metrics is a listening service that generates metrics about the state of Kubernetes objects through leveraging the Kubernetes API.

• Node_exporter is a Prometheus exporter for hardware and OS metrics exposed by *NIX kernels, written in Go is installed seperately in every GWEN and Ubuntu device. The node_exporter is designed to monitor the host system and it requires access to the host system so it’s not recommended to get deployed as a Docker container.

1.4.3.3. Place in architecture

Performance and Usage Diagnosis (PUD) enabler is located in the Application and Service layer of the ASSIST-IoT architecture that provides application logic, including data visualisation and user interaction services, data analytics capabilities, various kinds of data protection support, and data management logic. The PUD enabler is responsible to collect performance metrics from monitored targets.

PUD’s Prometheus scrapes metrics from instrumented jobs. It stores all scraped samples locally and runs rules over this data to either aggregate and record new time series from existing data. It works well for recording any purely numeric time series. It fits both machine-centric monitoring as well as monitoring of highly dynamic service-oriented architectures. In a world of microservices, its support for multi-dimensional data collection and querying is a particular strength. PUD’s Prometheus is designed for reliability, to be the system you go to during an outage to allow you to quickly diagnose problems. Each Prometheus server is standalone, not depending on network storage or other remote services. You can rely on it when other parts of your infrastructure are broken, and you do not need to setup extensive infrastructure to use it.

1.4.3.4. User guide

PUD’s Prometheus server provides a web UI for running basic queries located at http://<IP>:<NodePort>/. This is how it looks like in a web browser:

The “Table” tab is used to view the results of a query, while the “Graph” tab is used to create graphs based on a query.

If you want to see a list of metrics sources, go to the Status > Targets page. Here, you will find a list of all services that are being monitored, including the path at which the metrics are available. In this case, the default path /metrics is used.

If you’re curious to see how the metrics page looks like, head over to one of them by clicking one of the endpoint URLs.

The Prometheus server collects metrics and stores them in a time series database. Individual metrics are identified with names such as kube_pod_container_resource_requests. A metric may have a number of “labels” attached to it, to distinguish it from other similar sources of metrics. As an example, suppose kube_pod_container_resource_requests refers to the number of requested request resource by a container. It may have a label such as resource, which helps you inspect individual system resources by mentioning them.

In PromQL, an expression or subexpression should always evaluate to one of the following data types:

• Instant vector — It represents a time-varying value at a specific point of time.

• Range vector — it represents a time-varying value, over a period of time.

• Scalar — A simple numeric floating point value.

• String — A string value. String literals can be enclosed between single quotes, double quotes or backticks (`). However, escape sequences like n are only processed when double quotes are used.

For more about Querying please refer to Prometheus’ documentation to get started.

To dynamically add, update and delete scraping targets access PUD’s Prom-target-api swagger page http://<IP>:<NodePort>/docs. The updated targets can be seen in Prometheus’ server targets page.

PUD’s Grafana provides a powerful and flexible platform for creating visually appealing and informative dashboards to help you gain insights and monitor the whole system/architecture.

First the user needs to get logged in:

To access PUD’s Grafana Dashboard UI:

Port forward grafana’s pod to port 3000:

kubectl port-forward pude-grafana-6986754ffd-7gr62 3000

In PUD’s Grafana login page use:

Username: admin

To find the current password enter:

kubectl get secret pude-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo

To get kubernetes secrets and grafana’s secret name witch in our case is pude-grafana enter:

kubectl get secrets

To change your grafanas password enter:

kubectl exec -it <grafanas pod name> grafana-cli admin reset-admin-password <your reset password>

After login user should choose and add Prometheus data sourse in PUD’s Grafana.

By going to Settings > Add Data Source > Prometheus.

To set Prometheus’ URL under HTTP settings first find performanceandusagediagnosis-server clusterIP:

kubectl get services

• Copy and Paste the IP in the URL field.

• Save & Test

After choosing data source user should import new Dashboards for PUD’s Grafana.

• Go to Dashboards > + Import.

• Upload Dashboard’s json file or choose one from grafana.com.

• Load

Dashboards regarding Kube state metrics and Node_exporter can be found in PUD’s repository in grafana-dashboards directory.

By going to Dashboards user can access and manage all of his dashboards.

1.4.3.5. Prerequisites

• Kubernetes 1.16+

• Helm 3+

1.4.3.6. Installation

1.4.3.6.1. Performance and Usage Diagnosis (PUD) Enabler Installation

To install the chart with the release name pude for Ubuntu Architectures:

helm repo add assist-public-repo https://gitlab.assist-iot.eu/api/v4/projects/85/packages/helm/stable
helm install pude assist-public-repo assist-public-repo/performanceandusagediagnosis

Else you can also clone https://github.com/assist-iot/pud repo to your machine and install Performance and Usage Diagnosis Enabler.

Install Performance and Usage Diagnosis Enabler

helm install pude ./performanceandusagediagnosis

To check if the installation was successful run:

kubectl get pods

The result should show something like:

NAME                                                              READY   STATUS    RESTARTS          AGE
pude-grafana-6769c95d99-gz8q7                                     1/1     Running   0                 119s
pude-kube-state-metrics-5d764bf9d9-2p6r7                          1/1     Running   0                 119s
pude-performanceandusagediagnosis-prometheusesadapter-6d87pgkwt   1/1     Running   0                 119s
pude-performanceandusagediagnosis-server-579b59c787-6zngr         2/2     Running   0                 119s
pude-performanceandusagediagnosis-targetapi-68b74bb646-qhtnr      1/1     Running   0                 119s
pude-prometheus-node-exporter-fjr2k                               1/1     Running   0                 119s

Node_exporter Installation

Node_exporter is a Prometheus exporter for hardware and OS metrics exposed by *NIX kernels. It’s part of PUD’s helm chart but because is designed to monitor the host system it’s not recommended to deploy it as a Docker container because it requires access to the host system.

Node_exporter could also be installed separately on edge nodes and other devices in order to monitor their hardware and OS. Below is the installation process of Node_exporter as system service.

• Create a node_exporter user to run the node exporter service.

sudo useradd -rs /bin/false node_exporter

• Create a node_exporter service file under systemd.

sudo vi /etc/systemd/system/node_exporter.service

• Add the following service file content to the service file and save it.

[Unit]
Description=Node Exporter
After=network.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target

• Reload the system daemon and star the node exporter service.

sudo systemctl daemon-reload
sudo systemctl start node_exporter

• Check the node exporter status to make sure it is running in the active state.

sudo systemctl status node_exporter

• Enable the node exporter service to the system startup.

sudo systemctl enable node_exporter

Now, node exporter would be exporting metrics on port 9100.

1.4.3.7. Configuration options

The following table lists the configurable parameters of the Prometheus chart and their default values.

Parameter	Description	Default
nameOverride	string to partially override enabler.fullname template	""
fullnameOverride	string to fully override enabler.fullname template	""
enablerNodeSelector	Deploy all the components in specific K8s node(s)	{}
globalService	Cilium Multi-cluster global service.	false
server.enabled	whether prometheus server component is enabled	true
server.tier	prometheus server’s tier	external
server.replicaCount	prometheus server’s number of replicas	1
server.imagePullSecrets	prometheus server’s image pull secrets	[]
server.image.repository	prometheus server’s image repository	quay.io/prometheus/prometheus
server.image.tag	prometheus server’s image tag	"v2.36.2"
server.image.pullPolicy	prometheus server’s image pull policy	IfNotPresent
server.service.type	prometheus server’s service type	NodePort
server.service.ports.server.enabled	whether prometheus server’s port is enabled	true
server.service.ports.server.port	prometheus server’s port	9090
server.service.ports.server.targetPort	prometheus server’s targetPort	9090
server.service.ports.server.containerPort	prometheus server’s containerPort	9090
server.service.ports.server.nodePort	prometheus server’s nodePort	""
server.service.ports.server.protocol	prometheus server’s port protocol	TCP
server.resources	resource requests and limits	{}
server.autoscaling.enabled	whether prometheus server’s autoscaling is enabled	true
server.podSecurityContext	prometheus server’s pod security context	{}
server.securityContext	prometheus server’s security context	{}
server.envVars	prometheus server’s environmental variables	see values.yaml
server.configPath	prometheus server’s configPath	/etc/config/prometheus.yml
server.global.scrape_interval	how frequently prometheus server scrape targets by default	1m
server.global.scrape_interval	how long until a scrape request times out	10s
server.global.evaluation_interval	how frequently prometheus server evaluate rules	1m
server.remoteWrite	prometheus server remote write configuration	""
server.remoteRead	prometheus server remote read configuration	""
server.nodeSelector	prometheus server’s node selector	{}
server.tolerations	prometheus server’s tolerations	[]
server.affinity	prometheus server’s affinity	{}
alertmanager.enabled	whether alertmanager component is enabled	false
alertmanager.tier	alertmanager’s tier	external
alertmanager.replicaCount	alertmanager’s number of replicas	1
alertmanager.imagePullSecrets	alertmanager’s image pull secrets	[]
alertmanager.image.repository	alertmanager’s image repository	quay.io/prometheus/alertmanager
alertmanager.image.tag	alertmanager’s image tag	"v0.24.0"
alertmanager.image.pullPolicy	alertmanager’s image pull policy	IfNotPresent
alertmanager.service.type	alertmanager’s service type	ClusterIP
alertmanager.service.ports.alertmanager.port	alertmanager’s port	80
alertmanager.service.ports.alertmanager.targetPort	alertmanager’s targetPort	80
alertmanager.service.ports.alertmanager.containerPort	alertmanager’s containerPort	80
alertmanager.service.ports.alertmanager.nodePort	alertmanager’s nodePort	""
alertmanager.service.ports.alertmanager.protocol	alertmanager’s protocol	TCP
alertmanager.resources	resource requests and limits	{}
alertmanager.autoscaling.enabled	whether alertmanager’s autoscaling is enabled	true
alertmanager.podSecurityContext	alertmanager’s pod security context	{}
alertmanager.securityContext	alertmanager’s security context	{}
alertmanager.certificates	alertmanager’s sertificates	{}
alertmanager.envVars	alertmanager’s environmental variables	see values.yaml
alertmanager.nodeSelector	alertmanager’s node selector	{}
alertmanager.tolerations	alertmanager’s tolerations	[]
alertmanager.affinity	alertmanager’s affinity	{}
prometheusesadapter.enabled	whether prometheus-es-adapter component is enabled	true
prometheusesadapter.tier	prometheus-es-adapter’s tier	external
prometheusesadapter.replicaCount	prometheus-es-adapter’s number of replicas	1
prometheusesadapter.imagePullSecrets	prometheus-es-adapter’s image pull secrets	[]
prometheusesadapter.image.repository	prometheus-es-adapter’s image repository	gitlab.assist-iot.eu:5050/enablers-registry/public/pud/prometheus-es-adapter
prometheusesadapter.image.tag	prometheus-es-adapter’s image tag	"latest"
prometheusesadapter.image.pullPolicy	prometheus-es-adapter’s image pull policy	IfNotPresent
prometheusesadapter.service.type	prometheus-es-adapter’s service type	ClusterIP
prometheusesadapter.service.ports.pea8000.port	prometheus-es-adapter’s port	8000
prometheusesadapter.service.ports.pea8000.targetPort	prometheus-es-adapter’s targetPort	8000
prometheusesadapter.service.ports.pea8000.containerPort	prometheus-es-adapter’s containerPort	8000
prometheusesadapter.service.ports.pea8000.nodePort	prometheus-es-adapter’s nodePort	""
prometheusesadapter.service.ports.pea8000.protocol	prometheus-es-adapter’s protocol	TCP
prometheusesadapter.service.ports.pea8001.port	prometheus-es-adapter’s port	8001
prometheusesadapter.service.ports.pea8001.targetPort	prometheus-es-adapter’s targetPort	8001
prometheusesadapter.service.ports.pea8001.containerPort	prometheus-es-adapter’s containerPort	8001
prometheusesadapter.service.ports.pea8001.nodePort	prometheus-es-adapter’s nodePort	""
prometheusesadapter.service.ports.pea8001.protocol	prometheus-es-adapter’s protocol	TCP
prometheusesadapter.resources	resource requests and limits	{}
prometheusesadapter.autoscaling.enabled	whether prometheus-es-adapter’s autoscaling is enabled	true
prometheusesadapter.podSecurityContext	prometheus-es-adapter’s pod security context	{}
prometheusesadapter.securityContext	prometheus-es-adapter’s security context	{}
prometheusesadapter.certificates	prometheus-es-adapter’s sertificates	{}
prometheusesadapter.envVars	prometheus-es-adapter’s environmental variables	see values.yaml
prometheusesadapter.nodeSelector	prometheus-es-adapter’s node selector	{}
prometheusesadapter.tolerations	prometheus-es-adapter’s tolerations	[]
prometheusesadapter.affinity	prometheus-es-adapter’s affinity	{}
targetapi.enabled	whether prom-target-api component is enabled	true
targetapi.tier	prom-target-api’s tier	external
targetapi.replicaCount	prom-target-api’s number of replicas	1
targetapi.imagePullSecrets	prom-target-api’s image pull secrets	[]
targetapi.image.repository	prom-target-api’s image repository	gitlab.assist-iot.eu:5050/enablers-registry/public/pud/prom-target-api
targetapi.image.tag	prom-target-api’s image tag	"latest"
targetapi.image.pullPolicy	prom-target-api’s image pull policy	IfNotPresent
targetapi.service.type	prom-target-api’s service type	NodePort
targetapi.service.ports.targetapi.port	prom-target-api’s port	5000
targetapi.service.ports.targetapi.targetPort	prom-target-api’s targetPort	5000
targetapi.service.ports.targetapi.containerPort	prom-target-api’s containerPort	5000
targetapi.service.ports.targetapi.nodePort	prom-target-api’s nodePort	""
targetapi.service.ports.pea8000.protocol	prom-target-api’s protocol	TCP
targetapi.resources	resource requests and limits	{}
targetapi.autoscaling.enabled	whether prom-target-api’s autoscaling is enabled	true
targetapi.podSecurityContext	prom-target-api’s pod security context	{}
targetapi.securityContext	prom-target-api’s security context	{}
targetapi.certificates	prom-target-api’s sertificates	{}
targetapi.envVars	prom-target-api’s environmental variables	see values.yaml
targetapi.nodeSelector	prom-target-api’s node selector	{}
targetapi.tolerations	prom-target-api’s tolerations	[]
targetapi.affinity	prom-target-api’s affinity	{}
configmapReload.prometheus.name	configmapReload’s name	configmap-reload
configmapReload.prometheus.image.repository	configmapReload’s image repository	jimmidyson/configmap-reload
configmapReload.prometheus.image.tag	configmapReload’s image tag	v0.5.0
configmapReload.prometheus.image.pullPolicy	configmapReload’s image pullPolicy	IfNotPresent

The following table lists the configurable parameters of the Prometheus-elasticsearch-adapter chart and their default values.

Env Variables	Description	Default
ES_URL	Elasticsearch URL	http://localhost:9200
ES_USER	Elasticsearch User
ES_PASSWORD	Elasticsearch User Password
ES_WORKERS	Number of batch workers	1
ES_BATCH_MAX_AGE	Max period in seconds between bulk Elasticsearch insert operations	10
ES_BATCH_MAX_DOCS	Max items for bulk Elasticsearch insert operation	1000
ES_BATCH_MAX_SIZE	Max size in bytes for bulk Elasticsearch insert operation	4096
ES_ALIAS	Elasticsearch alias pointing to active write index	prom-metrics
ES_INDEX_DAILY	Create daily indexes and disable index rollover	false

See installation and configuration proccess in video.

1.4.3.8. Developer guide

1.4.3.8.1. PUD’s Prometheus Metrics & Exporters

Performance and Usage Diagnosis (PUD) Enabler follows an HTTP pull model: It scrapes performance metrics from endpoints routinely. Typically the abstraction layer between the application and PUD is an exporter, which takes application-formatted metrics and converts them to Prometheus metrics for consumption. Because PUD uses an HTTP pull model, the exporter typically provides an endpoint /metrics where the performance metrics can be scraped.

The relationship between Prometheus, the exporter, and the application in a Kubernetes environment can be visualized like this:

Metrics are served as plaintext. They are designed to be consumed either by PUD itself or by a scraper that is compatible with scraping a Prometheus client endpoint. The raw metrics can also be visualized in a browser by opening /metrics endpoint. Note that the metrics exposed on the /metrics endpoint reflect the current state of the application monitored.

The Prometheus metrics format is so widely adopted that it became an independent project: OpenMetrics, striving to make this metric format specification an industry standard.

1.4.3.8.2. Prometheus metrics naming

Generally metric names should allow someone who is familiar with Prometheus but not a particular system to make a good guess as to what a metric means. A metric named http_requests_total is not extremely useful - are these being measured as they come in, in some filter or when they get to the user’s code? And requests_total is even worse, what type of requests?

Metric names for applications should generally be prefixed by the exporter name, e.g. haproxy_up.

Metrics must use base units (e.g. seconds, bytes) and leave converting them to something more readable to graphing tools. No matter what units you end up using, the units in the metric name must match the units in use.

Prometheus metrics and label names are written in snake_case. Only [a-zA-Z0-9:_] are valid in metric names.

The _sum, _count, _bucket and _total suffixes are used by Summaries, Histograms and Counters. Unless you’re producing one of those, avoid these suffixes. _total is a convention for counters, you should use it if you’re using the COUNTER type. Prometheus metric format has a name combined with a series of labels or tags.

<metric name>{<label name>=<label value>, ...}

A time series with the metric name http_requests_total and the labels service=”service”, server=”pod50” and env=”production” could be written like this:

http_requests_total{service="service", server="pod50", env="production"}

You can associate any number of context-specific labels to every metric you submit. Imagine a typical metric like http_requests_per_second, every one of your web servers is emitting these metrics. You can then bundle the labels (or dimensions): - Web Server software (Nginx, Apache) - Environment (production, staging) - HTTP method (POST, GET) - Error code (404, 503) - HTTP response code (number) - Endpoint (/webapp1, /webapp2) - Datacenter zone (east, west)

Prometheus metrics text-based format is line oriented. Lines are separated by a line feed character (n). The last line must end with a line feed character. Empty lines are ignored. A metric is composed by several fields: - Metric name - Any number of labels (can be 0), represented as a key-value array - Current metric value - Optional metric timestamp

A Prometheus metric can be as simple as: http_requests 2

Or, including all the mentioned components: http_requests_total{method="post",code="400"}  3   1395066363000

Metric output is typically preceded with # HELP and # TYPE metadata lines.

The HELP string identifies the metric name and a brief description of it. The TYPE string identifies the type of metric. If there’s no TYPE before a metric, the metric is set to untyped. Everything else that starts with a # is parsed as a comment.

# HELP metric_name Description of the metric
# TYPE metric_name type
# Comment that's not parsed by prometheus
http_requests_total{method="post",code="400"}  3   1395066363000

1.4.3.8.3. Prometheus metrics client libraries

The Prometheus project maintains 4 official Prometheus metrics libraries written in Go, Java / Scala, Python, and Ruby. The Prometheus community has created many third-party libraries that you can use to instrument other languages (or just alternative implementations for the same language):

• Bash

• C++

• Common Lisp

• Elixir

• Erlang

• Haskell

• Lua for Nginx

• Lua for Tarantool

• .NET / C#

• Node.js

• Perl

• PHP

• Rust

1.4.3.8.4. Prometheus metrics / OpenMetrics types

Depending on what kind of information you want to collect and expose, you’ll have to use a different metric type. Here are your four choices available on the OpenMetrics specification:

Counter

This represents a cumulative metric that only increases over time, like the number of requests to an endpoint. Note: instead of using Counter to instrument decreasing values, use Gauges.

# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 3.7156890216e+10

Gauge

Gauges are instantaneous measurements of a value. They can be arbitrary values which will be recorded. Gauges represent a random value that can increase and decrease randomly such as the load of your system.

# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 73

Histogram

A histogram samples observations (usually things like request durations or response sizes) and counts them in configurable buckets. It also provides a sum of all observed values. A histogram with a base metric name of exposes multiple time series during a scrape:

# HELP http_request_duration_seconds request duration histogram
# TYPE http_request_duration_seconds histogram
http_request_duration_seconds_bucket{le="0.5"} 0
http_request_duration_seconds_bucket{le="1"} 1
http_request_duration_seconds_bucket{le="2"} 2
http_request_duration_seconds_bucket{le="3"} 3
http_request_duration_seconds_bucket{le="5"} 3
http_request_duration_seconds_bucket{le="+Inf"} 3
http_request_duration_seconds_sum 6
http_request_duration_seconds_count 3

Summary

Similar to a histogram, a summary samples observations (usually things like request durations and response sizes). While it also provides a total count of observations and a sum of all observed values, it calculates configurable quantiles over a sliding time window. A summary with a base metric name of also exposes multiple time series during a scrape:

More regarding OpenMetrics types

1.4.3.8.5. Prometheus exporters

Many popular server applications like Nginx or PostgreSQL are much older than the Prometheus metrics / OpenMetrics popularization. They usually have their own metrics formats and exposition methods. To work around this hurdle, the Prometheus community is creating and maintaining a vast collection of Prometheus exporters. An exporter is a “translator” or “adapter” program able to collect the server native metrics and re-publishing these metrics using the Prometheus metrics format and HTTP protocol transports. These small binaries can be co-located in the same container or pod executing the main server that is being monitored, or isolated in their own sidecar container and then you can collect the service metrics scraping the exporter that exposes and transforms them into Prometheus metrics.

There are a number of exporters that are maintained as part of the official Prometheus GitHub.

You might need to write your own exporter if…

• You’re using 3rd party software that doesn’t have an existing exporter already

• You want to generate Prometheus metrics from software that you have written

1.4.3.8.6. Example

Building a generic HTTP server metrics exporter in Python. By Nancy Chauhan: https://levelup.gitconnected.com/building-a-prometheus-exporter-8a4bbc3825f5

1.4.3.9. Version control and release

Prometheus v2.31.1

Prometheus-es-adapter v3.3

Prom-target-api v1.0

Grafana v9.1.1

kube-state-metrics v2.8.1

node_exporter v0.18.1

1.4.3.10. License

Copyright 2023 ICCS

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.