Skip to content

Observability — logs and metrics

DioscHub emits three things you monitor a running deployment with: structured logs to stdout, a Prometheus metrics endpoint, and health probes for your load balancer. This page covers each, and is clear about where the responsibility for sensitive data in your telemetry sits — with you.

DioscHub exposes Prometheus metrics at /api/metrics in the standard text exposition format. Metrics are event-driven: they are incremented as sessions start, tools run, and connections open, not sampled on a timer.

The families it exposes:

AreaMetrics
Sessionsdiosc_sessions_created_total, diosc_sessions_completed_total, diosc_sessions_active
MCP toolsdiosc_mcp_tool_calls_total, diosc_mcp_tool_duration_seconds, diosc_mcp_tool_retries_total, diosc_mcp_circuit_breaker_trips_total, diosc_mcp_auth_errors_total
Knowledge retrievaldiosc_rag_indexing_total, diosc_rag_chunks_indexed_total, diosc_rag_tokens_indexed_total
WebSocketdiosc_ws_connections_active, diosc_ws_disconnections_total, diosc_ws_rate_limited_total
Guardrailsdiosc_guardrail_input_blocked_total, diosc_guardrail_output_blocked_total
Problem reportsdiosc_problem_reports_total, diosc_problems_open, diosc_problem_webhook_deliveries_total, diosc_problem_webhook_delivery_failures_total
Processdiosc_process_uptime_seconds, diosc_process_heap_bytes, diosc_process_rss_bytes

/api/metrics takes a bearer token, METRICS_TOKEN, and fails closed:

METRICS_TOKENOutside dev modeIn dev mode (DIOSC_DEV_MODE=true)
Unset404: metrics are offOpen, no token needed
SetAuthorization: Bearer <token> required; anything else gets 401Same: the token is required

Use a random value of at least 32 characters (for example openssl rand -base64 48). A shorter token fails the boot gate outside dev mode. Then give your Prometheus job the token:

scrape_configs:
- job_name: dioschub
metrics_path: /api/metrics
authorization:
type: Bearer
credentials_file: /etc/prometheus/dioschub-metrics-token
static_configs:
- targets: ['dioschub.internal:3333']

The token protects the endpoint, but metrics are still operational telemetry, not a user-facing surface: keep /api/metrics reachable only by your monitoring where you can. See Production hardening.

There is also an authenticated admin metrics endpoint, /api/admin/metrics/realtime, that returns a JSON snapshot for the admin portal. It requires an admin session with analytics permission, so it is safe to leave reachable — but it is for the portal, not for scraping.

Problem reports and their webhook sinks add four metrics. Every label is a fixed value or an assistant id; none carries a user, session, sink, URL or delivery id, and a value outside the known set is recorded as unknown.

MetricTypeLabelsMeaning
diosc_problem_reports_totalcountersource (assistant_offer, user_command), assistant_assessment (defect, not_a_defect, unsure), assistant_idReports stored (one per approved report)
diosc_problems_opengaugenoneOpen problems across the hub
diosc_problem_webhook_deliveries_totalcounterevent, outcome (delivered, failed, cancelled)Webhook deliveries by final outcome, counted once each
diosc_problem_webhook_delivery_failures_totalcounterevent, reasonFailed delivery attempts, by reason: url_refused, redirect, timeout, network, http_4xx, http_5xx, http_429, template_invalid, payload_too_large, secret_unavailable

event is one of report.created, problem.created, problem.status_changed and report.erased. A rising diosc_problem_webhook_delivery_failures_total{reason="http_5xx"} means a sink’s receiver is struggling; a steady url_refused means the sink URL policy refuses the sink.

Three endpoints back load-balancer and orchestrator checks:

  • /api/health — overall health, including a database ping.
  • /api/health/ready — readiness, also database-backed; use it to gate traffic.
  • /api/health/live — a static liveness response; use it to decide whether to restart the container.

DioscHub logs to stdout, one structured line per event. In production the format is JSON; each line carries a context field naming the component and the event’s own fields. Ship stdout to your log aggregator — that is the primary sink.

You tune the log stream with environment variables:

VariableEffect
LOG_LEVELVerbosity (info in production, debug in development).
LOG_FORMATjson or pretty.
LOG_FILES_ENABLED, LOG_DIRAlso write rotating log files to a directory.

In production, file logging is on by default; set LOG_FILES_ENABLED=false to turn it off. When it is on, DioscHub writes rotating files (combined, error, audit, and conversation streams) under LOG_DIR — that directory is inside the container, so mount a volume if you want those files to survive a restart. For most deployments, shipping stdout to an aggregator is what you rely on.

What securing your telemetry leaves to you

Section titled “What securing your telemetry leaves to you”

DioscHub does not scrub its operational logs for you. Beyond a narrow denylist applied to conversation and tool-payload logging, the general application log stream is written as-is. Treat the log destination as sensitive, control verbosity with LOG_LEVEL, and apply redaction at your aggregator if your compliance posture requires it.

Auth credentials are a deliberate exception: by design they stay out of the log path, the same way BYOA keeps them out of the model’s context. That is the intent the code is built around — it is not a substitute for securing the log sink itself, which remains yours to protect.