**Thanos** is an open source system that extends [[Prometheus]] with unlimited retention, global query view, and high availability. It is the standard solution when a single Prometheus instance is insufficient — which, at hyperscaler scale, it always is. --- ### First Principle: Prometheus is a single-host tool. Production observability is multi-host. Thanos bridges that gap. A single Prometheus instance stores data locally and can only query its own scrapes. Thanos adds a sidecar to each Prometheus that ships data to object storage ([[MinIO]], [[Ceph]], or S3), a Query layer that fans out queries across multiple Prometheuses, and a Store Gateway that serves historical data directly from object storage. --- ### Key Considerations - **Sidecar Architecture**: A Thanos Sidecar runs alongside each Prometheus pod, uploading completed 2-hour blocks to object storage and exposing a gRPC Store API for the Querier. - **Querier**: The Thanos Querier provides a single PromQL endpoint that fans out queries across all sidecars and store gateways — giving a unified view across all clusters and time ranges. - **Store Gateway**: Reads historical metrics blocks directly from object storage, serving them through the Store API. This allows infinite retention without keeping data on Prometheus's local disk. - **Compactor**: Processes historical blocks in object storage — downsampling old data and applying retention policies. - **Rule Evaluation**: Thanos Ruler evaluates Prometheus recording rules and alerts against global data — across all Prometheuses — something a single Prometheus cannot do. - **vs [[Mimir]]**: Thanos is more modular and uses existing Prometheus instances. [[Mimir]] is Grafana's opinionated integrated solution. Thanos is more common in existing deployments; Mimir is better for greenfield. --- ### How It Fits ``` Multiple [[Prometheus]] instances (per cluster) → Thanos Sidecar (ships blocks to [[MinIO]]/S3) → Thanos Querier (unified PromQL endpoint) → Thanos Store Gateway (serves historical data) → [[Grafana]] (queries through Querier) ``` [[Prometheus]] | [[Mimir]] | [[Grafana]] | [[MinIO]] | [[Open Source Hyperscaler MoC]]