Initial commit: router front-door vLLM stack

Three always-running vLLM services (text TP=2 GPU0+1, ocr + embed on GPU2,
sleep mode) behind a FastAPI router that auto-wakes models on request.
Tiered idle (sleep 15 min / offload 3 h), depth-aware 503s with
Retry-After, persisted wake-intent recovery, admin API on 127.0.0.1:8010.
Routine control via vllmctl is pure HTTP — no docker on the request path.

Verified: 91 router unit tests + 15-test E2E on real hardware
(measurements in CALIBRATION.md; design record in
.claude/memory/router-front-door-plan.md).

Old nginx stack files removed before git init; design survives in
.claude/memory/sleep-mode-implementation-plan.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.claude/memory/MEMORY.md

@@ -0,0 +1,57 @@
+# vLLM Project Memory
+
+Index of memory files for this project.
+
+## Architecture & Design
+
+- [Router Front Door Plan v3.3](router-front-door-plan.md) — **IMPLEMENTED & VERIFIED 2026-08-17**: per-model vLLM services (all sleeping) + FastAPI router that auto-wakes on request. Pure-HTTP load/unload, no docker on the request path, replaced nginx. Two review rounds + full E2E.
+
+- [Front Door Proxy Design](front-door-proxy-design.md) — original research (vLLM Sleep Mode, Triton, vllm-proxy) that led here.
+
+- [Sleep Mode Implementation Plan v2](sleep-mode-implementation-plan.md) — the earlier Sleep Mode + nginx stack (implemented 2026-08-14, then superseded by the router plan).
+
+## Project Documentation
+
+- [`../README.md`](../README.md) — architecture, request behavior (503/Retry-After semantics), vllmctl, router tunables
+- [`../TODO.md`](../TODO.md) — implementation record, verification trail, known limitations, housekeeping
+- [`../CALIBRATION.md`](../CALIBRATION.md) — measured GPU footprints, wake latencies, backend quirks
+- [`../NOTES-2026-08-13.md`](../NOTES-2026-08-13.md) — 2026-08-13 rework + GPU P2P diagnosis
+- [`../NOTES-2026-08-17-gpu2-recheck.md`](../NOTES-2026-08-17-gpu2-recheck.md) — P2P fault moved (not fixed); root-cause analysis (VT-d peer-DMA translation); fix ladder for admin
+
+## Model Configuration (live)
+
+| Key | Served model | Service / GPUs | util |
+|-----|--------------|----------------|------|
+| text | `Qwen3.6-35B-A3B-FP8` | vllm-text, TP=2, GPU0+1 | 0.85, 262K ctx |
+| ocr | `OvisOCR2` | vllm-ocr, GPU2 | 0.10 |
+| embed | `Qwen3-Embedding-8B` | vllm-embed, GPU2 | 0.25 |
+
+Public API `:8000/v1` (all three always available, auto-wake); admin API
+`127.0.0.1:8010`; debug ports `127.0.0.1:8001-8003`.
+
+## Hard-won facts (do not regress)
+
+- **P2P corrupts data host-wide** (moves between GPUs across reboots; VT-d
+  peer-DMA platform fault). `NCCL_P2P_DISABLE=1` + `--disable-custom-all-reduce`
+  mandatory everywhere. GPU2 is fine for single-GPU services.
+- **vLLM backend quirks**: `/health` returns 200 while asleep (gate on
+  `/is_sleeping`); requests to a sleeping backend hang; requests between
+  `wake_up` and `reload_weights` return garbage; L1 wake needs `wake_up`
+  only; L1→L2 re-sleep is a no-op (wake-then-sleep to actually offload).
+- **`gpu_memory_utilization` counts TOTAL GPU memory per process**; keep
+  per-GPU sums ≤ 0.88 including CUDA contexts.
+- **Router state volume must be a named volume** (`router-state`), not a
+  bind mount — the non-root container user (10001) can't write a host-uid
+  dir, and the state store fails OPEN (fix silently inert).
+- Docker: `renbaibing` in docker group since 2026-08-14; shells older than
+  that need `sg docker -c "…"`.
+- OCR needs `--max-num-seqs 256` (GDN block limit at util 0.10).
+- FP8 model dir was corrupted by racing `hf download`s on 2026-08-14;
+  repaired + sha256-verified 2026-08-17 (corrupt copy kept, see TODO).
+
+## Housekeeping reminders
+
+- Delete `Qwen3.6-35B-A3B-FP8.corrupt-20260817` (43 GB, in MODEL_ROOT) once trusted.
+- Repo under git since 2026-08-17 (initial commit = router stack). The v2
+  nginx stack's runnable files were removed before git init; its design
+  survives in `sleep-mode-implementation-plan.md`.