Observability

Gatewyse exposes Prometheus metrics for scraping, ships a ready-to-import Grafana dashboard and alert rules, and can export OpenTelemetry traces, metrics, and logs over OTLP. The monitoring assets live under monitoring/ in the repository.

Prometheus metrics

The server exposes Prometheus metrics at GET /metrics. The endpoint is unauthenticated by default — put it behind network policy or a metrics-only listener in production.

All metric names use the aigw_ prefix, and every label set is bounded. Metrics are deliberately not labelled by tenant, user, or API key — that cardinality would explode the Prometheus series count, so per-tenant/user/key breakdowns live in Mongo usage records instead. Default Node.js process metrics are also collected under the aigw_ prefix (e.g. aigw_nodejs_eventloop_lag_p99_seconds).

Metric	Type	Labels	Description
aigw_http_requests_total	Counter	method, path, status	Total HTTP requests
aigw_http_request_duration_seconds	Histogram	method, path	HTTP request duration in seconds
aigw_provider_request_duration_seconds	Histogram	provider, model, status	Provider request duration in seconds
aigw_provider_errors_total	Counter	provider, error_type	Total provider errors
aigw_tokens_total	Counter	provider, model, type	Total tokens processed (type: input | output)
aigw_cost_usd_total	Counter	provider, model	Total estimated cost in USD
aigw_guard_blocks_total	Counter	guard_type, phase	Requests/responses blocked by a guard (phase: request | response)
aigw_budget_blocks_total	Counter	scope, limit_type	Requests blocked by a budget limit
aigw_cache_requests_total	Counter	result	Cache lookups by result (hit | semantic_hit | miss)
aigw_auth_failures_total	Counter	reason	Authentication failures by reason (bounded error-code set)
aigw_rate_limit_rejections_total	Counter	level	Requests rejected by a rate limiter (ip | apiKey | tenant | daily)
aigw_active_streams	Gauge	—	Number of active streaming connections
aigw_stream_ttft_seconds	Histogram	provider	Streaming time-to-first-token in seconds
aigw_queue_depth	Gauge	queue, state	BullMQ job counts (state: waiting | active | delayed | failed)
aigw_circuit_breaker_state	Gauge	provider	Circuit breaker state (0=closed, 1=half_open, 2=open)

URL paths are normalized before being used as label values — ObjectId-like segments and UUIDs are replaced with :id and query strings are stripped — to prevent label explosion.

Scraping

monitoring/prometheus/scrape.example.yml contains a scrape configuration and rule_files block to merge into your prometheus.yml. The primary Prometheus surface is the server’s /metrics endpoint:

scrape_configs:
  - job_name: gatewyse-server
    metrics_path: /metrics
    static_configs:
      - targets: ['gatewyse-server:3000']
        labels:
          service: gatewyse-server

The worker exports its telemetry over OTLP rather than a Prometheus endpoint, so scrape it only if you add a Prometheus exporter to it.

Grafana dashboard

monitoring/grafana/gatewyse-overview.json is a ready-to-import dashboard. Import it via Dashboards → New → Import, then select your Prometheus data source for the DS_PROMETHEUS input.

Panels cover:

• HTTP request rate by status and HTTP latency (p50 / p95 / p99)
• Provider latency p95 and provider error rate, by provider
• Token throughput by direction and estimated spend (USD/hour) by provider
• Active streams and streaming TTFT p95 by provider
• Circuit-breaker state per provider
• Guard blocks by guard type and budget blocks by scope
• Cache hit ratio
• Queue depth by queue/state, and auth failures & rate-limit rejections

Alerting

monitoring/prometheus/alerts.yml defines alerting rules — load it via rule_files: in prometheus.yml. Every expression references a metric the gateway actually exports; thresholds are conservative starting points, tune per deployment SLO.

Alert	Severity	Fires when
GatewyseTargetDown	critical	A gatewyse.* scrape target has been down for 2m
GatewyseCircuitBreakerOpen	warning	A provider’s circuit breaker has been open for 1m
GatewyseHighHttpErrorRate	critical	5xx responses exceed 5% over 5m
GatewyseHighProviderErrorRate	warning	A provider’s error rate exceeds 10% over 5m
GatewyseHighRequestLatencyP95	warning	HTTP p95 latency exceeds 2s for 10m
GatewyseHighProviderLatencyP95	warning	A provider’s p95 latency exceeds 30s for 10m
GatewyseBudgetBlockSpike	info	Budget blocks sustained above 1/s for 5m
GatewyseGuardBlockSpike	info	A guard blocks above 1/s for 5m
GatewyseAuthFailureSpike	warning	Auth failures sustained above 5/s for 5m
GatewyseQueueBacklog	warning	A queue has >1000 waiting jobs for 10m
GatewyseHighEventLoopLag	warning	Node.js event-loop lag p99 exceeds 250ms for 10m

OpenTelemetry (traces, metrics, logs)

Both the server (service.name = ai-gateway) and the worker (service.name = ai-gateway-worker) can export OTLP traces, metrics, and logs. The entire pipeline is gated on OTEL_EXPORTER_OTLP_ENDPOINT: when it is unset (the default) the OpenTelemetry bootstrap is a no-op with zero overhead, and the manual provider spans resolve to no-op spans.

When enabled:

• Traces — auto-instrumentation for HTTP, Express (server), Mongoose, ioredis, and undici (outbound HTTP). Manual provider spans are emitted from the base adapter regardless of auto-instrumentation.
• Metrics — the auto-instrumentations emit HTTP/DB/Redis metrics via OTLP alongside the Prometheus /metrics endpoint.
• Logs — Winston records are forwarded into the OTLP logs pipeline with active trace context (trace_id / span_id) injected onto each record.

Both resources are tagged service.namespace = gatewyse. The exporters append /v1/traces, /v1/metrics, and /v1/logs to the configured endpoint base URL.

For full ESM auto-instrumentation, start each process with the OTel bootstrap imported first:

node --import ./dist/telemetry/otel.js ./dist/index.js

Environment variables

Variable	Description
OTEL_EXPORTER_OTLP_ENDPOINT	Collector base URL (e.g. http://otel-collector:4318); /v1/{traces,metrics,logs} are appended. Unset = telemetry disabled.
OTEL_SERVICE_NAME	Overrides service.name (defaults ai-gateway / ai-gateway-worker).
OTEL_METRIC_EXPORT_INTERVAL_MS	Metric export interval in milliseconds (default 30000).

Health and readiness endpoints

Endpoint	Purpose
GET /health	Liveness — the process is up
GET /ready	Readiness — dependencies (MongoDB, Redis) are reachable and the service can accept traffic
GET /metrics	Prometheus metrics (see above)

Use /health for liveness probes and /ready for readiness probes and load-balancer health checks.