Incident record for the first level-2 wake failure (cumem allocator,
text hard-down ~16 min, container restart restored it) plus the
caller-side findings from remote-access testing: 503 Retry-After
verified present on every server path, and the model-alias 404 quirk.
Co-Authored-By: Claude <noreply@anthropic.com>
Three always-running vLLM services (text TP=2 GPU0+1, ocr + embed on GPU2,
sleep mode) behind a FastAPI router that auto-wakes models on request.
Tiered idle (sleep 15 min / offload 3 h), depth-aware 503s with
Retry-After, persisted wake-intent recovery, admin API on 127.0.0.1:8010.
Routine control via vllmctl is pure HTTP — no docker on the request path.
Verified: 91 router unit tests + 15-test E2E on real hardware
(measurements in CALIBRATION.md; design record in
.claude/memory/router-front-door-plan.md).
Old nginx stack files removed before git init; design survives in
.claude/memory/sleep-mode-implementation-plan.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
bingandClaude