Labs / Lab 13

Put model access behind a localhost broker

Keep the host pointed at 127.0.0.1 while the broker handles credential lookup, request checks, and upstream forwarding.

What you'll build

A local HTTP broker that stands between a host and the real credential.

By the end of this lab you will have a tiny broker process listening on 127.0.0.1. A host can send model requests to that local address, and the broker decides whether the request is allowed, where it should go upstream, and whether it should attach a bearer token.

The teaching version starts in dry_run mode, so you can see the full request and response cycle without a real API key or a real outbound call. Once that shape makes sense, the same broker can pull a secret from an environment variable, 1Password, or Azure Key Vault.

Run it

cd ai_ecosystem_labs
python3 13-local-broker/broker.py --config 13-local-broker/broker_config.dry-run.json
Starting here? Quick setup 
git clone https://github.com/BanditF/ai_ecosystem_labs
cd ai_ecosystem_labs
python3 13-local-broker/broker.py --config 13-local-broker/broker_config.dry-run.json

Requires Python 3.8+. No extra packages needed for this lab.

Time guide. Setup: 5–10 min. Working through it: 25–45 min, especially if you compare dry-run mode with the real secret-handling path.

Why this piece exists

The point is to move trust to a narrower place.

If the host holds the raw provider key, every config file, shell session, or tool wrapper that talks to the model becomes part of the secret's blast radius. A localhost broker changes that shape. The host only knows how to call a local HTTP interface, and the broker is the only component that needs to know how to fetch or attach the real credential.

That does not make localhost magically safe. It does give you a cleaner trust boundary: allowlisted paths, optional caller authentication, request size limits, and one place to swap secret backends without changing host code. Real systems often do this with an internal gateway or backend service. This lab is the smallest version of that idea.

Real-world analog: your editor, CLI, or agent host talks to a small local gateway, and that gateway talks to OpenAI or another provider. The host is holding a capability, not the root credential.

The code

broker.py

Walk through it

Four things worth noticing.

The host never holds the real key

The host talks to http://127.0.0.1:8791, not directly to the provider. In proxy mode, the broker resolves the secret and adds the Authorization: Bearer ... header itself. That keeps the provider credential out of host config files and out of most of the places where local tools normally leak secrets.

Local auth still matters

The stronger example configs add client_token_env, which means callers must present X-Broker-Token. The code checks it with hmac.compare_digest() before forwarding. The goal is simple: another process on the same machine should not get to impersonate the intended host just because it can reach localhost.

Dry-run mode teaches the whole flow safely

broker_config.dry-run.json sets mode to dry_run and secret_source to none. The broker still parses the request, checks the path allowlist, builds the forward target, and returns a structured response. You can see the interface clearly before you wire in real credentials.

Config backends are swappable

The interface stays the same while the secret source changes. In the env example, the broker reads OPENAI_API_KEY. In the 1Password example, it runs op read .... In the Azure Key Vault example, it runs az keyvault secret show .... The host code does not care which one you picked.

Expected output

What a successful dry run looks like.

Your file paths may appear as absolute paths depending on where you run the script — that's expected.

When the broker starts:

{
  "ok": true,
  "listen": "http://127.0.0.1:8791",
  "mode": "dry_run",
  "allowed_paths": [
    "/v1/chat/completions",
    "/v1/responses",
    "/v1/embeddings"
  ]
}

Then a dry-run request to an allowed path returns a mock forward description instead of calling a provider:

{
  "ok": true,
  "mode": "dry_run",
  "forward": {
    "url": "https://api.openai.example/v1/responses",
    "path": "/v1/responses",
    "secret_source": "none",
    "would_attach_bearer_token": false
  },
  "request": {
    "prompt": "hello"
  }
}

The broker also appends an event to labs/13-local-broker/events.jsonl so you can see that the request was accepted and logged:

{"time":"2025-01-01T00:00:00Z","kind":"request","path":"/v1/responses","ok":true,"status":200,"mode":"dry_run"}

Try this

Three things to try before you call this finished.

  1. Run the dry-run broker and send it a test request. In one terminal, start the broker. In another, run curl -X POST http://127.0.0.1:8791/v1/responses -H "Content-Type: application/json" -d '{"prompt":"hello"}'. Read the JSON response and then open labs/13-local-broker/events.jsonl. You should see the same request cycle reflected in the log.
  2. Try a request the broker should reject. Switch to one of the proxy example configs, set BROKER_CLIENT_TOKEN, and send a request without the X-Broker-Token header or with the wrong value. The broker should return 401. Then send a request to a path not listed in allowed_paths with a valid X-Broker-Token header. That returns 403.
  3. Compare the secret backends. Read broker_config.env.example.json and compare it to broker_config.1password.example.json. The broker still listens on the same local interface and forwards the same kinds of requests. The only meaningful difference is where it fetches the provider key from.

Concepts behind this

If you want the broader security framing, read API key security.

If you want to connect this back to tool-using loops and host behavior, read agents.

Next in the sequence

Continue to Lab 14 to compare prompt shapes side by side, or go back to all labs if you want to jump around. If you want to reconnect this security pattern to a real host, Lab 03b is still a good side trip.