Guard Proxy: `multimind serve`

Your app (OpenAI SDK) ──> multimind serve ──> upstream (OpenAI, Groq, Ollama, ...)
                          │
                          ├─ redacts PII in requests (mask/hash/remove)
                          ├─ blocks requests carrying forbidden PII types (400)
                          ├─ enforces a spend ceiling (429 when exceeded)
                          ├─ scans model output for PII (optional)
                          └─ writes a JSONL audit trail (never raw content)

​

60-second quickstart

1. Start the proxy (requires pip install 'multimind-sdk[gateway]'):

export OPENAI_API_KEY=sk-...
multimind serve --port 8400 --strategy mask --block-on ssn,credit_card \
    --budget 5.00 --audit-log audit.jsonl

multimind serve --port 8400 --upstream ollama

2. Change one line in your existing app:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8400/v1", api_key="unused")

# Everything else stays exactly the same:
resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize this: my SSN is 212-45-6789"}],
)

​

What it exposes

• 400 type compliance_violation, code pii_blocked — a --block-on PII type was found in the request (includes blocked_types).
• 429 type/code budget_exceeded — the session budget is spent (includes spent and max_cost).
• 502 type upstream_error, code upstream_unreachable — the upstream could not be reached. Upstream 4xx/5xx responses are passed through unchanged. On a streaming request these errors arrive as an SSE error event (code upstream_unreachable or upstream_error) followed by [DONE], since headers have already been sent.

​

Configuration

from multimind.gateway.guard_proxy import ProxySettings, create_app

app = create_app(ProxySettings(upstream="ollama", block_on=("ssn",), budget=5.0))
# serve `app` with uvicorn, or mount it inside a larger FastAPI app

# Other constructors: ProxySettings.from_env(**overrides) reads the
# MULTIMIND_PROXY_* variables; ProxySettings.from_file("guardrails.json")
# layers a dashboard-authored file on top of them.

​

docker-compose

services:
  guard-proxy:
    image: your-multimind-image
    command: multimind serve --host 0.0.0.0 --port 8400 --block-on ssn,credit_card
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - MULTIMIND_PROXY_AUDIT_LOG=/data/audit.jsonl
    ports:
      - "8400:8400"
    volumes:
      - ./audit:/data

​

Honest limitations

• Latency: every request takes one extra local hop plus regex scanning. Overhead is small (milliseconds) but not zero.
• Streaming is slightly buffered: to catch PII split across SSE chunks, the proxy holds back a 64-character overlap window before emitting each chunk. Output arrives marginally behind the upstream, and chunk boundaries are re-segmented (total content is preserved).
• Regex-based detection: the built-in PIIDetector covers common structured PII (emails, SSNs, cards, keys, …). It is not an NLP entity recognizer and will not catch free-form PII like names or addresses.
• Budget accuracy: costs use real token usage when the upstream reports it and a chars/4 estimate otherwise (always for streams). Models outside the built-in OpenAI pricing table are recorded as unpriced ($0) unless you set MULTIMIND_PROXY_COST_PER_TOKEN, so the budget cannot bite there. The budget is per-process (“session”) and resets on restart.
• Chat surface: string content and {"type": "text"} multimodal parts are scanned; image/audio parts pass through unscanned. Tool-call arguments in responses are not currently scanned.
• Not authentication: the proxy does not authenticate its own callers; bind it to localhost or put it behind your own auth in production.