6.4Instrumenting Application Metrics

# metrics_setup.py -- Application metrics with Prometheus client
from prometheus_client import Counter, Histogram, Gauge, start_http_server

# Request metrics
request_count = Counter(
    'http_requests_total',
    'Total HTTP requests',
    ['method', 'path', 'status']
)

request_duration = Histogram(
    'http_request_duration_seconds',
    'HTTP request duration in seconds',
    ['method', 'path'],
    buckets=[0.01, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0]
)

# Business metrics
orders_created = Counter(
    'orders_created_total',
    'Total orders created',
    ['payment_method', 'status']
)

active_sessions = Gauge(
    'active_sessions',
    'Number of active user sessions'
)

# Start metrics endpoint
start_http_server(8000)  # /metrics endpoint on port 8000

The USE and RED Methods

Two frameworks for choosing what to measure:

USE Method (for infrastructure):

• Utilization: How full is the resource? (CPU %, memory %, disk %)
• Saturation: How much extra work is queued? (queue depth, thread pool usage)
• Errors: How often does work fail? (I/O errors, connection failures)

RED Method (for services):

• Rate: How many requests per second?
• Errors: How many of those requests fail?
• Duration: How long do those requests take?