Service Telemetry

Service telemetry provides essential real-time data on system health, performance, and usage. It enables:

  • Monitoring and Diagnostics: Early detection of issues for quick resolution.
  • Performance Optimization: Identifying bottlenecks to improve efficiency.
  • Usage Insights: Understanding user behavior to guide improvements.
  • Security: Detecting suspicious activities and ensuring compliance.
  • Scalability: Predicting and managing resource demands.

In the Inference server, the following telemetry options are available:

  • Prometheus metrics
  • Docker container metrics provided by Docker daemon

Prometheus in Inference Server

To enable metrics, set the environmental variable ENABLE_PROMETHEUS=True in your docker container:

docker run -p 9001:9001 -e ENABLE_PROMETHEUS=True roboflow/roboflow-inference-server-cpu

Then use the GET /metrics endpoint to fetch the metrics in Python:

import requests

result = requests.get("http://127.0.0.1:9001/metrics")
result.raise_for_status()

print(result.text)

or using cURL:

curl http://127.0.0.1:9001/metrics

Docker Container Metrics

Warning

This feature relies on Docker daemon socket exposure inside the container. This allows exposing docker container resource utilization metrics without the need for a "supervisor" service, but may be seen as a security concern. This option is disabled by default. Please acknowledge potential security risks before enabling this option.

To expose container metrics you need to run the docker container with the Inference server in a specific way:

docker run -p 9001:9001 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -e DOCKER_SOCKET_PATH=/var/run/docker.sock \
  roboflow/roboflow-inference-server-cpu

Explanation:

  • The -v flag mounts the docker daemon socket from your host (typically /var/run/docker.sock) into a location inside the container.
  • The -e DOCKER_SOCKET_PATH exposes the environmental variable with the location of the docker daemon socket inside the container.

Then you can reach the GET /device/stats endpoint using cURL:

curl http://127.0.0.1:9001/device/stats

or using Python:

import requests

result = requests.get("http://127.0.0.1:9001/device/stats")
result.raise_for_status()

print(result.json())