vllm-frontdoor/.claude/memory/sleep-mode-implementation-plan.md

# Implementation Plan: vLLM Sleep Mode + nginx Front Door

**Date:** 2026-08-14  
**Status:** Revised (v2) — Critical issues from review addressed

## Changelog (v2)

**Fixed Critical Issues:**
1. ✅ Healthcheck dependency — changed to `service_started` (doesn't require healthy)
2. ✅ nginx upstream host — changed from `127.0.0.1:8001` to `vllm:8000`
3. ✅ Port binding clarified — vLLM: `127.0.0.1:8001:8000` (localhost only), nginx: external 8000
4. ✅ Added `--enable-sleep-mode` flag to EXTRA_ARGS
5. ✅ Implemented idle watcher update with `/sleep` calls

**Important Concerns Addressed:**
- Added note about `/metrics` exposure
- Clarified in-flight request handling during `/sleep`
- Updated wake time estimate to be more realistic
- Removed redundant `return 403` after `deny all`

---

## Overview

Transform the current vLLM setup from full container teardown (`docker compose down`) to vLLM Sleep Mode, with nginx as a security front door.

### Current Architecture

```
Client → [vLLM Container :8000]
           ↓
    down = docker compose down (full teardown)
    up   = docker compose up (cold start, 2-10 min)
```

### Target Architecture

```
Internet → [nginx :8000] → [vLLM :8000 (Docker network) ]
                           ↓
                           [vLLM :8001 (localhost only) ]
                           ↓
                      sleep = POST /sleep (model hibernates, GPU freed)
                      wake  = POST /wake_up (model resumes, ~5-15s for 35B)
```

### Port Binding Clarification

| Port | Access | Purpose |
|------|--------|---------|
| `8000` (nginx) | External (0.0.0.0:8000) | Public API, clients connect here |
| `8001` (vLLM) | Localhost only (127.0.0.1:8001) | Admin access for `vllmctl`, sleep/wake endpoints |
| `8000` (Docker network) | Internal | nginx → vLLM communication within Docker network |

---

## Phase 1: Enable vLLM Sleep Mode

### 1.1 Update `compose.yml`

**Critical changes:**

1. Add `VLLM_SERVER_DEV_MODE=1` to environment (required for Sleep Mode endpoints)
2. Add `--enable-sleep-mode` to EXTRA_ARGS (this was missing in v1!)
3. Expose vLLM on localhost only: `127.0.0.1:8001:8000`
4. Keep vLLM accessible on Docker network at port 8000

**Before:**
```yaml
services:
  vllm:
    image: vllm/vllm-openai:${VLLM_VERSION:-v0.27.1}
    container_name: vllm
    restart: unless-stopped
    ports:
      - "${VLLM_PORT:-8000}:8000"
    volumes:
      - ${MODEL_ROOT:-/data/home/renbaibing/huggingface}:/models:ro
    ipc: host
    gpus: all
    environment:
      - HF_TOKEN=${HF_TOKEN:-}
      - MODEL_NAME=${MODEL_NAME}
      - TP_SIZE=${TP_SIZE:-2}
      - MAX_MODEL_LEN=${MAX_MODEL_LEN:-32768}
      - GPU_MEM_UTIL=${GPU_MEM_UTIL:-}
      - EXTRA_ARGS=${EXTRA_ARGS:-}
      - NCCL_P2P_DISABLE=${NCCL_P2P_DISABLE:-1}
    healthcheck:
      test: ["CMD-SHELL", "python3 -c \"import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)\" || exit 1"]
      interval: 15s
      timeout: 10s
      retries: 8
      start_period: 1200s
```

**After:**
```yaml
services:
  vllm:
    image: vllm/vllm-openai:${VLLM_VERSION:-v0.27.1}
    container_name: vllm
    restart: unless-stopped
    
    # Two port bindings:
    # 1. Docker network port 8000 (internal, for nginx)
    # 2. Localhost port 8001 (admin access for vllmctl)
    ports:
      - "127.0.0.1:8001:8000"  # Admin access - localhost only
      expose:
        - "8000"  # Docker network - for nginx
    
    volumes:
      - ${MODEL_ROOT:-/data/home/renbaibing/huggingface}:/models:ro
    ipc: host
    gpus: all
    
    environment:
      - VLLM_SERVER_DEV_MODE=1  # Required for Sleep Mode endpoints
      - HF_TOKEN=${HF_TOKEN:-}
      - MODEL_NAME=${MODEL_NAME}
      - TP_SIZE=${TP_SIZE:-2}
      - MAX_MODEL_LEN=${MAX_MODEL_LEN:-32768}
      - GPU_MEM_UTIL=${GPU_MEM_UTIL:-}
      # EXTRA_ARGS MUST include --enable-sleep-mode
      - EXTRA_ARGS=${EXTRA_ARGS:- --enable-sleep-mode}
      - NCCL_P2P_DISABLE=${NCCL_P2P_DISABLE:-1}
    
    # Healthcheck stays the same - container will be "unhealthy" during sleep
    # This is OK - nginx uses service_started, not service_healthy
    healthcheck:
      test: ["CMD-SHELL", "python3 -c \"import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)\" || exit 1"]
      interval: 15s
      timeout: 10s
      retries: 8
      start_period: 1200s
```

**Rationale:** 
- `VLLM_SERVER_DEV_MODE=1` enables dev endpoints including `/sleep`, `/wake_up`, `/is_sleeping`
- `--enable-sleep-mode` in EXTRA_ARGS is REQUIRED for sleep functionality (this was the critical missing piece in v1)
- Port `127.0.0.1:8001` gives `vllmctl` localhost-only access to admin endpoints
- `expose: - "8000"` makes port 8000 available on Docker network for nginx
- Healthcheck will fail during sleep, but that's OK — nginx uses `service_started` condition

### 1.2 Update `vllmctl`

**First, update the API URL:**

```bash
# Change at the top of vllmctl:
VLLM_PORT="$(env_get VLLM_PORT 8000)"
API="http://127.0.0.1:8001"  # Direct to vLLM localhost port, bypasses nginx
```

**Modify `cmd_down()`:**

```bash
cmd_down() {
    say 'Putting vLLM to sleep (frees GPU memory, keeps server alive)...'
    
    # Try to sleep the model (level 2 = discard weights, minimal RAM)
    if curl -fs -X POST "$API/sleep?level=2" >/dev/null 2>&1; then
        ok "Model is sleeping. Server still running."
        
        # Wait a moment for sleep to complete
        sleep 2
        
        # Verify sleep state
        local sleeping
        sleeping="$(curl -fs "$API/is_sleeping" 2>/dev/null)"
        if [ "$sleeping" = "true" ]; then
            ok "Confirmed: Model is in sleep state."
        else
            warn "Sleep state unclear - check with: curl $API/is_sleeping"
        fi
    else
        warn "Sleep request failed — server may not be ready. Falling back to full stop."
        dcompose down -t 20 || return 1
    fi
    
    # Show GPU memory after sleep
    local used
    used="$(nvidia-smi --query-gpu=index,memory.used --format=csv,noheader,nounits 2>/dev/null | awk -F', *' '{printf "GPU%s:%sMiB ", $1, $2}')"
    say "GPU memory now: $used"
}
```

**Rationale:**
- `level=2` discards weights entirely (minimal RAM usage)
- Wake time for 35B model: ~5-15s (more realistic than the 0.8-2.6s estimate)
- Adds sleep verification to confirm the operation worked
- Falls back to full stop if sleep fails

**Modify `cmd_up()`:**

```bash
cmd_up() {
    local model="${1:-}"
    if [ -n "$model" ]; then
        if [ ! -d "$MODEL_ROOT/$model" ]; then
            err "Model '$model' not found in $MODEL_ROOT"
            err "Available:"; cmd_list
            return 1
        fi
        env_set MODEL_NAME "$model"
    fi
    model="$(env_get MODEL_NAME)"
    [ -n "$model" ] || { err 'No MODEL_NAME set in .env'; return 1; }

    local st
    st="$(container_state)"
    
    # Case 1: Container not running at all
    case "$st" in
        absent*|exited*|dead*)
            say "Starting vLLM with model '${C_CYAN}$model${C_OFF}' (TP=$(env_get TP_SIZE 2), Sleep Mode enabled)…"
            dcompose up -d || return 1
            wait_for_ready "$model"
            return 0
            ;;
    esac

    # Case 2: Container running - check if sleeping or different model
    local sleeping loaded
    sleeping="$(curl -fs "$API/is_sleeping" 2>/dev/null)"
    loaded="$(loaded_model)"
    
    # Model already loaded and serving
    if [ "$sleeping" != "true" ] && [ "$loaded" = "$model" ]; then
        ok "Already serving '$model'."
        return 0
    fi

    # Model is sleeping - wake it up
    if [ "$sleeping" = "true" ]; then
        say "Model is sleeping. Waking up '${C_CYAN}$model${C_OFF}'…"
        # Level 2 sleep requires reload_weights after wake_up
        curl -fs -X POST "$API/wake_up" >/dev/null 2>&1 || {
            err "Wake-up request failed."
            return 1
        }
        # Reload weights (required for level 2 sleep)
        curl -fs -X POST "$API/collective_rpc" \
            -H 'Content-Type: application/json' \
            -d '{"method":"reload_weights"}' >/dev/null 2>&1 || {
            warn "reload_weights request failed - model may still wake up"
        }
        # Reset prefix cache (required for level 2 sleep)
        curl -fs -X POST "$API/reset_prefix_cache" >/dev/null 2>&1 || {
            warn "reset_prefix_cache failed - non-critical"
        }
        # Wait for model to be ready
        wait_for_ready "$model"
        ok "Model '$model' is awake and ready."
        return 0
    fi

    # Different model loaded - need to switch
    if [ -n "$loaded" ] && [ "$loaded" != "$model" ]; then
        say "Switching from '${C_YELLOW}$loaded${C_OFF}' to '${C_CYAN}$model${C_OFF}'…"
        say "Restarting container with new model (this will take a few minutes)…"
        dcompose down -t 20
        sleep 2
        dcompose up -d || return 1
        wait_for_ready "$model"
        ok "Now serving '$model'."
        return 0
    fi

    # Shouldn't reach here, but handle gracefully
    err "Unexpected state - container running but no model loaded. Try: ./vllmctl restart"
    return 1
}
```

**Rationale:**
- Now handles the sleeping state properly
- For level 2 sleep, calls the required sequence: `/wake_up` → `reload_weights` → `reset_prefix_cache`
- Keeps container restart for model switching (cleaner than in-place reload)
- Better error messages and state handling

**Modify `cmd_status()`:**

```bash
cmd_status() {
    local st loaded cfg sleeping
    st="$(container_state)"
    cfg="$(env_get MODEL_NAME)"

    say "${C_CYAN}Container:${C_OFF} $st"
    
    # Check if sleeping first (loaded_model won't work when sleeping)
    sleeping="$(curl -fs "$API/is_sleeping" 2>/dev/null)"
    
    if [ "$sleeping" = "true" ]; then
        say "${C_CYAN}Serving:${C_OFF}   ${C_DIM}model sleeping${C_OFF}  (configured: $cfg)"
    else
        loaded="$(loaded_model)"
        if [ -n "$loaded" ]; then
            say "${C_CYAN}Serving:${C_OFF}   ${C_GREEN}$loaded${C_OFF} at http://127.0.0.1:${VLLM_PORT}/v1"
        else
            say "${C_CYAN}Serving:${C_OFF}   ${C_DIM}nothing loaded${C_OFF}  (configured: $cfg)"
        fi
    fi

    say "${C_CYAN}GPU:${C_OFF}"
    nvidia-smi --query-gpu=index,name,memory.used,memory.total,utilization.gpu \
        --format=csv,noheader 2>/dev/null | sed 's/^/    /'

    # External access (nginx)
    say "${C_CYAN}Public API:${C_OFF} http://$(hostname -f | head -1):${VLLM_PORT}/v1"
    say "${C_CYAN}Admin API:${C_OFF} $API (localhost only)"

    # Idle watcher status
    local pidfile="$ROOT/.idle.pid"
    if [ -f "$pidfile" ] && kill -0 "$(cat "$pidfile" 2>/dev/null)" 2>/dev/null; then
        say "${C_CYAN}Idle-watch:${C_OFF} active (pid $(cat "$pidfile), $(cat "$ROOT/.idle.minutes" 2>/dev/null) min timeout)"
    else
        say "${C_CYAN}Idle-watch:${C_OFF} ${C_DIM}off${C_OFF}  (enable: ./vllmctl idle-watch on [minutes])"
    fi
}
```

**Modify `idle_loop()` — CRITICAL FIX:**

```bash
idle_loop() {
    local timeout_min="$1"
    local timeout_s=$(( timeout_min * 60 ))
    local last_active=0
    say "[idle-watch] started: will auto-sleep after ${timeout_min} min without requests (poll 30s)"
    
    # Use internal API for sleep calls
    local SLEEP_API="http://127.0.0.1:8001"
    
    while true; do
        sleep 30
        
        # Check if container is running at all
        local st
        st="$(container_state)"
        case "$st" in
            absent*|exited*|dead*)
                # Container not running - nothing to do
                last_active=0
                continue
                ;;
        esac
        
        # Check if already sleeping
        local sleeping
        sleeping="$(curl -fs "$SLEEP_API/is_sleeping" 2>/dev/null)"
        if [ "$sleeping" = "true" ]; then
            # Already sleeping, nothing to do
            continue
        fi
        
        # Check for activity using metrics
        local idle_s='' run_wait=0
        idle_s="$(curl -fs -m 5 "$SLEEP_API/metrics" 2>/dev/null \
            | awk '/^vllm:time_since_last_request_seconds/ {print $2; exit}')"
        
        if [ -z "$idle_s" ] || [ "$idle_s" = "+Inf" ] || [ "$idle_s" = "NaN" ]; then
            # Fall back: count running/waiting requests
            run_wait="$(curl -fs -m 5 "$SLEEP_API/metrics" 2>/dev/null \
                | awk '/^vllm:num_requests_(running|waiting)/ {s+=$2} END {print s+0}')"
            if [ "${run_wait:-0}" != "0" ]; then
                last_active="$(date +%s)"
                continue
            fi
            [ "$last_active" = 0 ] && last_active="$(date +%s)"
            idle_s=$(( $(date +%s) - last_active ))
        fi
        
        # Integer compare (strip possible decimals)
        local idle_i=${idle_s%%.*}
        if [ -n "$idle_i" ] && [ "$idle_i" -ge "$timeout_s" ] 2>/dev/null; then
            say "[idle-watch] idle for ${idle_i}s ≥ ${timeout_s}s → putting model to sleep"
            if curl -fs -X POST "$SLEEP_API/sleep?level=2" >/dev/null 2>&1; then
                say "[idle-watch] model is now sleeping at $(date '+%F %T')"
            else
                warn '[idle-watch] sleep request failed - check if server is responsive'
            fi
            last_active=0
        fi
    done
}
```

**Rationale:**
- Uses internal API (`:8001`) for all calls
- Checks if container is running before attempting API calls
- Checks if already sleeping (avoid redundant sleep calls)
- Calls `/sleep?level=2` instead of `dcompose down`
- Graceful error handling if server is unresponsive

### 1.3 Update `.env`

No changes needed — everything is in `compose.yml` and `vllmctl`.

---

## Phase 2: Add nginx Front Door

### 2.1 Create `nginx.conf`

```nginx
events {
    worker_connections 1024;
}

http {
    # Rate limiting (optional, can be commented out)
    limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;

    # Upstream using Docker service name - CRITICAL FIX
    upstream vllm {
        server vllm:8000;  # Docker service name, not 127.0.0.1
        keepalive 32;
    }

    server {
        listen 8000;
        server_name _;

        # Block dev endpoints — deny all is sufficient
        location /sleep {
            deny all;
        }
        location /wake_up {
            deny all;
        }
        location /is_sleeping {
            deny all;
        }
        location /collective_rpc {
            deny all;
        }
        location /reset_prefix_cache {
            deny all;
        }

        # Main API proxy
        location / {
            limit_req zone=api_limit burst=20 nodelay;
            
            proxy_pass http://vllm;
            proxy_http_version 1.1;
            
            # Headers for WebSocket/streaming support
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            
            # Timeouts for LLM inference (increased from v1)
            proxy_read_timeout 900s;
            proxy_send_timeout 900s;
            proxy_connect_timeout 10s;
            
            # Disable buffering for streaming
            proxy_buffering off;
        }

        # Health endpoint (for health checks, allows unhealthy during sleep)
        location /health {
            proxy_pass http://vllm/health;
            access_log off;
            # Don't fail if unhealthy - vLLM can be sleeping
            proxy_next_upstream error timeout http_502 http_503 http_504;
        }

        # Metrics endpoint (WARNING: exposed without auth)
        # Consider adding authentication if this becomes publicly accessible
        location /metrics {
            proxy_pass http://vllm/metrics;
        }
    }
}
```

**Rationale:**
- Fixed: `server vllm:8000` uses Docker service name (was `127.0.0.1:8001` in v1)
- Removed redundant `return 403` after `deny all`
- Increased timeouts to 900s for long-running inference
- Added `proxy_next_upstream` for health endpoint tolerance
- Added warning comment about `/metrics` exposure

### 2.2 Update `compose.yml`

Add nginx service with CRITICAL FIX to dependency:

```yaml
services:
  nginx:
    image: nginx:alpine
    container_name: vllm-nginx
    restart: unless-stopped
    ports:
      - "${VLLM_PORT:-8000}:8000"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    # CRITICAL FIX: Use service_started, not service_healthy
    # vLLM healthcheck fails during sleep, but that's OK
    depends_on:
      vllm:
        condition: service_started  # Changed from service_healthy
    networks:
      - default

  vllm:
    # ... (as shown in Phase 1.1)
```

**Rationale:**
- **CRITICAL FIX**: Changed from `service_healthy` to `service_started`
- This allows nginx to start even when vLLM healthcheck fails (during sleep)
- nginx will still return 502 if vLLM is completely down, which is correct behavior

### 2.3 Port Binding Summary

After all changes, the port layout is:

| From | To | Who Can Access | Purpose |
|-----|-----|----------------|---------|
| `0.0.0.0:8000` | nginx:8000 | Everyone | Public API |
| nginx | vllm:8000 | Docker only | nginx→vLLM |
| `127.0.0.1:8001` | vllm:8000 | Localhost only | Admin (`vllmctl`) |

**Security Model:**
- External clients hit nginx on port 8000
- nginx blocks `/sleep`, `/wake_up`, `/is_sleeping`, etc.
- `vllmctl` uses localhost:8001 to bypass nginx and access admin endpoints
- vLLM containers can talk to each other on Docker network at vllm:8000

---

## Phase 3: Testing & Validation

### 3.1 Test Sleep Mode

```bash
# 1. Start model
./vllmctl up Qwen3.6-35B-A3B

# 2. Verify serving through nginx
curl http://localhost:8000/v1/models
# Should show: {"object":"list","data":[{"id":"Qwen3.6-35B-A3B",...}]}

# 3. Put to sleep
./vllmctl down
# Should show: "Model is sleeping. Server still running."

# 4. Check GPU memory freed
nvidia-smi
# GPU memory should be significantly lower

# 5. Verify external API blocked - dev endpoints
curl http://localhost:8000/sleep
# Should return: 403 Forbidden

# 6. Verify internal API works
curl http://localhost:8001/is_sleeping
# Should return: true

# 7. Test public API during sleep
curl http://localhost:8000/v1/models
# Should return error or timeout (model is sleeping)

# 8. Wake up
./vllmctl up
# Should wake up the existing model

# 9. Verify serving again
curl http://localhost:8000/v1/models
# Should work again
```

### 3.2 Test Idle Watcher

```bash
# Enable short idle timeout for testing
./vllmctl idle-watch on 1

# Wait 1 minute, then check status
./vllmctl status
# Should show: "model sleeping"

# Test that wake-up works
./vllmctl up
./vllmctl status
# Should show: serving the model

# Turn off when done
./vllmctl idle-watch off
```

### 3.3 Test Error Handling

```bash
# 1. Stop vLLM container
docker compose stop vllm

# 2. Try request through nginx
curl http://localhost:8000/v1/models
# Should get: 502 Bad Gateway

# 3. Start vLLM again
docker compose start vllm

# 4. Verify recovery
curl http://localhost:8000/v1/models
# Should work again
```

### 3.4 Test In-Flight Request Handling

**Note:** vLLM Sleep Mode will allow in-flight requests to complete before sleeping. Test:

```bash
# Start a long-running request in background
curl -N http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen3.6-35B-A3B","prompt":"Tell me a long story","max_tokens":500}' &
CURL_PID=$!

# Immediately put to sleep
./vllmctl down

# The request should complete (may take a moment)
wait $CURL_PID
echo "Request completed with exit code: $?"
```

---

## Phase 4: Optional Enhancements

### 4.1 Protect /metrics Endpoint

If `/metrics` exposure is a concern:

```nginx
location /metrics {
    # Optional: basic auth
    # auth_basic "Metrics";
    # auth_basic_user_file /etc/nginx/.htpasswd;
    
    # Or restrict to localhost
    # allow 127.0.0.1;
    # deny all;
    
    proxy_pass http://vllm/metrics;
}
```

### 4.2 Friendly Error When Model Is Sleeping

Add Lua-based checking (requires nginx with Lua module):

```nginx
# This requires nginx-lua or OpenResty - can add later if needed
```

### 4.3 Request Logging

Enable access logging for debugging:

```nginx
http {
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;
    # ...
}
```

And mount log volume in compose.yml:

```yaml
nginx:
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./logs/nginx:/var/log/nginx
```

---

## Migration Steps

1. **Backup current setup:**
   ```bash
   cp compose.yml compose.yml.bak
   cp vllmctl vllmctl.bak
   ```

2. **Update `compose.yml`:**
   - Add nginx service
   - Update vLLM service with new ports and environment
   - Add `--enable-sleep-mode` to EXTRA_ARGS

3. **Create `nginx.conf`:**
   - Copy the config above
   - Adjust as needed

4. **Update `vllmctl`:**
   - Change API URL to `:8001`
   - Modify `cmd_down()`, `cmd_up()`, `cmd_status()`, `idle_loop()`
   - Use the code blocks from Phase 1.2

5. **Test:**
   ```bash
   docker compose down
   docker compose up -d
   ./vllmctl status
   ```

6. **Rollback if needed:**
   ```bash
   docker compose down
   cp compose.yml.bak compose.yml
   cp vllmctl.bak vllmctl
   docker compose up -d
   ```

---

## Security Considerations

1. **Dev endpoints are protected** — nginx blocks `/sleep`, `/wake_up`, `/is_sleeping`, `/collective_rpc`, `/reset_prefix_cache`
2. **Internal access only** — `vllmctl` uses port 8001 which is bound to localhost only
3. **Container isolation** — vLLM and nginx are in the same Docker network
4. **WARNING: /metrics is exposed** — Consider adding authentication if deployment becomes public
5. **VLLM_SERVER_DEV_MODE=1** — This enables dev endpoints; ensure nginx properly blocks them

---

## Performance Impact

- **nginx overhead**: Minimal (~1-2ms latency, sub-1% CPU)
- **Sleep Mode wake**: ~5-15s for level 2 with 35B model (realistic estimate)
- **Memory**: Level 2 sleep uses minimal CPU RAM, frees ~90% GPU memory
- **In-flight requests**: vLLM allows completion before sleep

---

## Open Questions (Resolved)

1. ✅ **Should idle watcher call `/sleep` directly?**
   - **RESOLVED**: Yes, updated `idle_loop()` to call `/sleep?level=2` instead of `dcompose down`

2. **Should we optimize model switching?**
   - Current: Restart container on model switch (clean, reliable)
   - Could optimize later: Use `/sleep` + `/wake_up` + `reload_weights` for faster switches
   - Decision: Start with restart, optimize later if needed

3. ⚠️ **What about `/metrics` endpoint?**
   - Currently accessible through nginx (no auth)
   - **Decision**: Leave open for now, add auth later if deployment becomes public

---

## Estimated Effort (Updated)

- Phase 1 (vLLM Sleep Mode): ~45 minutes (increased due to more comprehensive changes)
- Phase 2 (nginx): ~30 minutes
- Phase 3 (Testing): ~45 minutes (more thorough testing)
- **Total**: ~2 hours (updated from 1.5 hours)

---

## Next Steps

Once you approve this revised plan, I'll:
1. Create the modified files (`compose.yml`, `nginx.conf`, `vllmctl`)
2. Test in a worktree or provide step-by-step instructions
3. Document rollback procedure

**All critical issues from the first review have been addressed.**

Ready to proceed?