Initial commit: router front-door vLLM stack

Three always-running vLLM services (text TP=2 GPU0+1, ocr + embed on GPU2,
sleep mode) behind a FastAPI router that auto-wakes models on request.
Tiered idle (sleep 15 min / offload 3 h), depth-aware 503s with
Retry-After, persisted wake-intent recovery, admin API on 127.0.0.1:8010.
Routine control via vllmctl is pure HTTP — no docker on the request path.

Verified: 91 router unit tests + 15-test E2E on real hardware
(measurements in CALIBRATION.md; design record in
.claude/memory/router-front-door-plan.md).

Old nginx stack files removed before git init; design survives in
.claude/memory/sleep-mode-implementation-plan.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

GPU-report.txt

@@ -0,0 +1,69 @@
+GPU/IOMMU fault report — v2, 2026-08-17
+(supersedes the 2026-08-14 report; the fault has MOVED, not healed)
+
+> Three NVIDIA A800-SXM4-80GB on CPU root ports (no PCIe switch):
+>   GPU0 = 0000:3d:00.0  serial 1324522063353
+>   GPU1 = 0000:63:00.0  serial 1324522063362
+>   GPU2 = 0000:ab:00.0  serial 1324522063359
+> Driver 595.71.05, kernel 6.8.0-137-generic (changed from 6.8.0-117 at the
+> Aug-14 reboot), VBIOS 92.00.A4.00.02 on all three.
+
+PROBLEM
+
+GPU<->GPU P2P DMA corrupts data. Corruption always follows the SOURCE GPU:
+
+    Source GPU | 2026-08-13 | 2026-08-17 (after reboot + kernel change)
+    -----------+------------+-------------
+    GPU0       | CORRUPT    | CORRUPT
+    GPU1       | clean      | CORRUPT
+    GPU2       | CORRUPT    | clean
+
+The Aug-14 reboot did not fix anything: GPU0 has been a corrupt source both
+times, and the failure relocated from GPU2's DMA path to GPU1's. Reproducible
+test: /data/home/renbaibing/vllm/diag/p2p_check.py (run inside any CUDA
+container, e.g. vllm/vllm-openai:v0.27.1). Result: 16,777,215 of 16,777,216
+elements wrong on the corrupt paths, bit-stable across runs.
+
+ROOT-CAUSE ASSESSMENT (platform translation path, NOT bad GPUs)
+
+- All hardware counters clean on all three GPUs: ECC volatile AND aggregate
+  zero, zero remapped rows, zero retired pages, links x16 Gen4.
+- Per-GPU H2D/D2H copies and matmul PASS on all three GPUs. Only
+  *peer*-addressed DMA corrupts.
+- Deterministic all-elements-wrong corruption = data written to wrong
+  addresses (translation fault), not random bit errors from failing silicon.
+- VT-d is ENABLED: 14 active DMAR units, /proc/cmdline has no "iommu=pt" —
+  every peer write crosses VT-d translation. There are no PCIe switches, so
+  all P2P funnels through the root complex + IOMMU.
+- Kernel evidence, all pointing at GPU0 (3d:00) as the initiator:
+  - Jul 6:  Xid 31 CE2 MMU faults (FAULT_PDE, VIRT_WRITE) on 3d:00
+  - Aug 13: DMAR: [DMA Write NO_PASID] device [3d:00.0] fault reason 0x71
+    "Present bit in first-level paging entry is clear" (hundreds)
+- All NVLink links report inActive on all three SXM4 boards (topo shows
+  PCIe-only: NODE/SYS). SXM4 boards exist for NVLink — links being down at
+  3 GPUs populated is itself worth investigating.
+
+ASK (in priority order)
+
+1. Decisive test: add "intel_iommu=on iommu=pt" to the kernel command line,
+   reboot, re-run diag/p2p_check.py. All 6 ordered pairs must print OK.
+   Clean result = root cause proven (VT-d peer-DMA translation).
+2. Investigate why NVLink is inactive (SBIOS setting / baseboard population /
+   link training). If NVLink comes up, NCCL stops using the PCIe P2P path
+   entirely and this bug stops mattering for our workload.
+3. Check ACS on the three GPU root ports; consider SBIOS/BMC firmware and a
+   driver branch matched to kernel 6.8.0-137 (driver + kernel both changed at
+   the same reboot that relocated the fault).
+4. Only if corruption survives iommu=pt: slot-swap two GPUs. Corruption
+   following the serial = bad board; following the slot = bad fabric.
+   Serials are recorded above for that comparison.
+
+IMPACT
+
+Tensor-parallel inference is unusable across GPUs without the software
+workaround (NCCL_P2P_DISABLE=1 + --disable-custom-all-reduce, i.e. NCCL
+stages through host memory). Measured cost at our TP=2 workload: none
+(~146 tok/s). Single-GPU workloads are unaffected and verified clean.
+Please re-run diag/p2p_check.py after ANY reboot or admin change and record
+the matrix; also re-record "nvidia-smi --query-gpu=index,serial,pci.bus_id"
+to track board vs slot.