Initial commit: router front-door vLLM stack
Three always-running vLLM services (text TP=2 GPU0+1, ocr + embed on GPU2, sleep mode) behind a FastAPI router that auto-wakes models on request. Tiered idle (sleep 15 min / offload 3 h), depth-aware 503s with Retry-After, persisted wake-intent recovery, admin API on 127.0.0.1:8010. Routine control via vllmctl is pure HTTP — no docker on the request path. Verified: 91 router unit tests + 15-test E2E on real hardware (measurements in CALIBRATION.md; design record in .claude/memory/router-front-door-plan.md). Old nginx stack files removed before git init; design survives in .claude/memory/sleep-mode-implementation-plan.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
19
.gitignore
vendored
Normal file
19
.gitignore
vendored
Normal file
@@ -0,0 +1,19 @@
|
||||
# secrets / local env — .env.example is the template
|
||||
.env
|
||||
.user.env
|
||||
|
||||
# Claude local settings (memory/ is committed, settings are not)
|
||||
.claude/settings.local.json
|
||||
|
||||
# ephemera
|
||||
*.bak
|
||||
*.log
|
||||
.idle.*
|
||||
router-data/
|
||||
GPU-report.local.txt
|
||||
|
||||
# python
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.pytest_cache/
|
||||
.venv/
|
||||
Reference in New Issue
Block a user