Metric Aggregator

Metric Aggregator uses Prometheus as the metric aggregator to scrape and store metrics from annotated pods. The Prometheus server is pre-configured to discover and scrape metrics from pods with the specified annotations. Below are the steps to access and query metrics:

1. Access Prometheus UI

Open the Prometheus web interface using the provided URL and provided credentials.

• Check target status at status -> Targets

• Verify that the pod is listed and the status is "UP".

2. Explore Metrics

Use the "Graph" tab to enter PromQL queries and visualize metrics.

• PromQL query example:

vllm:gpu_cache_usage_perc{job="prod-cluster-1", model_name="meta-llama/Llama-3.2-3B- Instruct"}