Observability Has a Cardinality Budget
Telemetry becomes infrastructure. Model useful signal against ingestion, storage, and query cost as dimensions grow.
Telemetry cardinality is discussed in architecture reviews and post-mortems, often as a reliability topic or a FinOps exercise. The conversation stops at policy slides. Constrange's Observe → Measure → Model series treats it as an engineering economics problem: what does the system actually do, what does that work cost, and what model explains the gap between intent and invoice.
Observe: count active series per metric and team. Cardinality explosions arrive quietly in the ingest bill.
Measure: dollars per million samples, query latency at p95, and dashboard time-to-answer during incidents.
Model: insight scales sub-linearly with dimensions; cost scales linearly or worse. There is a budget.
Telemetry as infrastructure
Metrics, logs, and traces are not free side effects of shipping code. They are storage systems with query engines and retention policies. High-cardinality labels — user ids, session ids, unbounded route templates — turn observability into a cost centre that competes with production workloads.
The cardinality budget
Assign each team a monthly cardinality budget: max active series, max ingest gigabytes, max custom metrics. Exceeding budget requires justification — usually an incident-driven temporary lift with expiry.
| Example | Metric safe? | |
|---|---|---|
| Low | service, region, status | Yes |
| Medium | route template, tenant tier | With caps |
| High | user_id, request_id | Traces/logs only, sampled |
| Unbounded | free-text error message | Never in metrics |
Sampling is a design choice
Trace sampling, log filtering, and metric aggregation are not admissions of failure. They are how you keep signal dense enough to use. Model sampling rate against incident debuggability — 1% may be fine until it is not.
“Dashboards you cannot afford to query are observability theatre.”Constrange
Questions for your next review
- What would we see if we measured cardinality budget for one week?
- Which dependency owns the majority of the cost or delay?
- What policy or architecture assumption does the data contradict?
- What is the smallest experiment that would change our model?
The organisations that improve cardinality budget are not those with the best slogans. They are those that observe honestly, measure without vanity metrics, and update the model when the bill or the trace disagrees with the diagram.
Frequently asked questions
What is cardinality?
The number of unique time series — often driven by high-cardinality labels like user id or request id.
How do I set a budget?
Cap labels per metric, sample traces, and measure ingest cost per team per month.
Does more telemetry mean more insight?
Only until query latency and cost make dashboards unusable. Signal-to-spend matters.
What labels are safe?
Low-cardinality: service, region, route, status class. Avoid unbounded ids in metrics.