/ runtime

A live view of the network.

This is the operator's view what our own on-call sees. Numbers move in real time. Nothing is cached. Nothing is polished for effect.

/ health
all systems operationalus-west-2 · normaleu-west-1 · normalap-northeast-1 · elevatedupdated · 00:21:27 UTC
requests / s
–
4,820
workers online
–
214
p50 latency
–
41ms
tokens / s
–
12,400
queue depth
–
0
target 0
gpu util · fleet
–
68%
credits burned · min
–
182 cr
stream throughput
–
9,200
/ regions

Traffic by region.

Each region is a set of independent workers under a shared router. The mesh rebalances continuously.

US-WEST-242 workersUS-EAST-158 workersSA-EAST-112 workersEU-WEST-134 workersEU-CENTRAL-128 workersAF-SOUTH-16 workersME-CENTRAL-18 workersAP-SOUTH-114 workersAP-SOUTHEAST-118 workersAP-NORTHEAST-122 workersAP-SOUTHEAST-210 workers
/ latency

Latency, unbucketed.

Live p50 · p95 · p99 across the network. The gap between percentiles is the fingerprint of the system.

p50
38ms
p95
164ms
p99
240ms
/ workers

Fleet state.

A sample of active workers. Real fleet is larger and rotates constantly.

workerregionmodelgpustatereq / s
wrk_a19fus-west-2orvex-1H100● serving66
wrk_c72eus-east-1orvex-1-miniL40S● serving34
wrk_9d10eu-west-1orvex-1H100● warm79
wrk_88baeu-central-1embed-1L4● serving75
wrk_4501ap-northeast-1orvex-1H100● elevated24
wrk_ff02ap-southeast-1image-1A100● serving57
wrk_2bebsa-east-1orvex-1-miniL40S● warm78