Except there was a gap. Every streaming call cost $0.00 in the ledger. Silently. No error, no warning – just wrong numbers.

I found the gap, fixed it, and got the PR merged. This is the story of how streaming cost tracking actually works.

What AgentBudget does

AgentBudget is an open-source SDK that puts a hard dollar limit on AI agent sessions. Two lines, no infrastructure:

1import agentbudget
2agentbudget.init("$5.00")

It works by monkey-patching Completions.create (OpenAI) and Messages.create (Anthropic) at init time. Every call goes through a wrapper that looks up the model cost, accumulates spend, and raises BudgetExhausted when the limit is hit.

The non-streaming path worked fine. The streaming path did not.

The bug

When stream=True, the patched method receives a Stream object back from the SDK – not a completed response with token counts. The original code had no handling for this case. It silently passed the stream through untracked.

Every streaming call registered as $0.00 in the ledger. Any agent using streaming could blow past its budget with zero resistance. The circuit breaker was blind to half its traffic.

Why streaming is harder to cost

Non-streaming is straightforward: the response comes back with a usage object containing prompt_tokens and completion_tokens. Multiply by the per-token rate, done.

Streaming returns a sequence of chunks. Token counts are not available upfront – they only appear after the stream is exhausted. For OpenAI specifically, usage data on the final chunk is opt-in: you need stream_options={"include_usage": True} in the request, otherwise the final chunk carries no usage data and there is nothing to cost.

Anthropic handles it differently. Usage data is always present, split across two events: message_start carries input tokens, message_delta carries output tokens at the end.

The fix: transparent wrapper classes

The solution is four wrapper classes that intercept the stream, yield every chunk unchanged, and record cost after the iterator is exhausted:

Each wrapper is fully transparent to the caller. The existing code pattern – for chunk in stream: or async for chunk in stream: or with client.stream(...) as stream: – continues to work without any changes.

The wrapper accumulates token counts as chunks arrive, then records the cost in the session ledger once the stream ends.

For OpenAI:

 1class _OpenAIStreamWrapper:
 2    def __init__(self, stream, session, model):
 3        self._stream = stream
 4        self._session = session
 5        self._model = model
 6
 7    def __iter__(self):
 8        for chunk in self._stream:
 9            # Track usage from the final chunk
10            if chunk.usage:
11                cost = price(self._model, chunk.usage)
12                self._session.record(cost)
13            yield chunk
14
15    # Full protocol: __enter__, __exit__, close()

For Anthropic, the accumulation spans two event types:

 1class _AnthropicStreamWrapper:
 2    def __iter__(self):
 3        input_tokens = 0
 4        output_tokens = 0
 5        for event in self._stream:
 6            if event.type == "message_start":
 7                input_tokens = event.message.usage.input_tokens
 8            elif event.type == "message_delta":
 9                output_tokens = event.usage.output_tokens
10            yield event
11        cost = price(self._model, input_tokens, output_tokens)
12        self._session.record(cost)

The detection of whether a response is a stream happens via isinstance checks guarded by ImportError, so the wrapper degrades gracefully if only one SDK is installed.

Implementing the full protocol

The tricky part is not the cost tracking logic – it is making the wrapper implement the full stream protocol correctly.

Python streams support multiple usage patterns: iterator (for chunk in stream), context manager (with stream), async iterator, async context manager, and explicit close()/aclose(). Missing any of these means certain callers will break with an AttributeError or a resource leak.

The wrapper needs to implement all of them:

 1# Sync: iterator + context manager + close
 2def __iter__(self): ...
 3def __enter__(self): return self
 4def __exit__(self, *args): self.close()
 5def close(self): self._stream.close()
 6
 7# Async: async iterator + async context manager + aclose
 8async def __aiter__(self): ...
 9async def __aenter__(self): return self
10async def __aexit__(self, *args): await self.aclose()
11async def aclose(self): await self._stream.aclose()

Getting this wrong is invisible in unit tests with mocks but breaks immediately against real SDK objects.

Tests

The PR added 19 tests covering:

• Cost recorded correctly after stream exhaustion
• Chunks with no usage data skipped without error
• Passthrough identity – every chunk the caller sees is unchanged
• BudgetExhausted propagates correctly mid-stream
• Both for chunk in stream: and with client.stream() as stream: patterns
• close() and aclose() delegation to the underlying stream
• Drop-in mode (agentbudget.init()) wrapping end-to-end
• No-session noop – if no budget session is active, streaming is fully transparent

All 129 pre-existing tests continued to pass. 148 total passing at merge.

The known limitation

OpenAI streaming requires the caller to pass stream_options={"include_usage": True}. Without it, the final chunk carries no usage data, and the cost is silently $0.

This is OpenAI’s API behaviour – usage data on streams is opt-in. The limitation is documented in the module docstring. A future improvement could auto-inject the option at the patch layer, but that requires modifying the outbound request, which is a bigger change than this PR intended to be.

Anthropic has no equivalent limitation – usage is always present.

What I learned

Transparent wrappers need the full protocol. It is tempting to implement just __iter__ and call it done. But real callers use context managers, explicit close, and async variants. Implementing the minimum breaks production code in ways that are hard to debug.

Streaming costs are provider-specific. OpenAI and Anthropic have different conventions for where usage data appears in the stream. Any SDK that claims to support both needs to handle both separately.

Test against real SDK objects, not just mocks. The isinstance detection and chunk structure only behave correctly against live SDK classes. Mock-based tests passed for the wrong reasons until I verified against an actual installed SDK.

Links

• PR #6 – feat: track costs for streaming OpenAI and Anthropic responses
• AgentBudget on GitHub
• PyPI

This is the story of setting that up in one session, and the parts that went sideways.

The Stack

Four moving parts:

• Bifrost – LLM gateway already running on localhost:24242, routing to Ollama cloud models
• Hermes Agent – open-source AI agent framework by Nous Research, pointed at Bifrost
• Ollama – cloud relay for gemma4 and qwen3.5 cloud models via ollama.com
• Camoufox – Firefox fork with TLS fingerprint spoofing for scraping tasks

The goal was to have Hermes run as a Telegram bot so I could assign it tasks from my phone, with all model traffic flowing through Bifrost so routing stays centralized.

Bifrost First

Bifrost was already up on port 24242 with Ollama configured. The only addition needed was an ollama-cloud provider pointing at https://ollama.com rather than localhost. This is a non-obvious distinction.

The trap: Ollama’s OpenAI-compatible endpoint (/v1/chat/completions) does not work for cloud models. It hangs. Cloud models only work through Ollama’s native /api/chat endpoint. Pointing Bifrost directly at https://ollama.com instead of localhost:11434 bypasses the broken compat layer and hits the right path.

Once the provider was registered and OLLAMA_API_KEY was set in .zshrc, both cloud models confirmed working:

1ollama pull gemma4:31b-cloud
2ollama pull qwen3.5:cloud

Installing Hermes

1curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash

That puts everything under ~/.hermes/. The setup wizard runs next:

1hermes setup

It walks through provider selection interactively. I pointed it at the Bifrost custom endpoint (http://localhost:24242/v1) and picked ollama-cloud/gemma4:31b-cloud as the default model.

Key config paths:

~/.hermes/config.yaml    -- main settings
~/.hermes/.env           -- secrets and API keys
~/.hermes/skills/        -- reusable agent procedures
~/.hermes/sessions/      -- full conversation history (searchable)

After setup, a quick smoke test confirmed the agent was up and routing through Bifrost correctly:

1hermes chat -q "what model are you using and where does traffic go"

Context Window

The default context length was under 20k tokens. Ollama cloud models support much larger windows. One config set:

1hermes config set model.context_length 200000

Takes effect on the next session restart.

Telegram Bot

This is where Hermes gets genuinely useful. The gateway turns it into a bot that accepts tasks from any platform you connect.

Create a bot via BotFather, get the token, then add it to ~/.hermes/.env:

1TELEGRAM_BOT_TOKEN=<your-bot-token>
2TELEGRAM_ALLOWED_USERS=<your-numeric-telegram-id>

TELEGRAM_ALLOWED_USERS takes the numeric user ID, not the username. Lock it down. Anyone who finds the bot URL can send it commands if this is not set.

Start the gateway:

1hermes gateway run

For persistent background operation, install it as a service:

1hermes gateway install
2hermes gateway start

A set of aliases in .zshrc makes it easier to manage day to day:

1alias hermes-gateway-start="hermes gateway start"
2alias hermes-gateway-stop="hermes gateway stop"
3alias hermes-gateway-log="tail -f ~/.hermes/logs/gateway.log"

First run hit a missing Python dependency and failed silently. The fix was running hermes gateway run in the foreground once to see the real error, installing the missing package, then moving back to the service.

Camoufox

Camoufox is a Firefox fork built for scraping. It spoofs TLS fingerprints so sites see a legitimate browser rather than a Playwright or CDP signature. It exposes a CDP endpoint that Hermes’s browser tool connects to.

1git clone https://github.com/daijro/camoufox
2cd camoufox && npm install
3python -m camoufox fetch   # ~300MB download

The camoufox fetch step downloads the actual browser binary and does not run automatically via postinstall. If you skip it, the first start fails silently. Run it manually and wait for the download to finish before starting the server.

Start it on port 9377:

1python -m camoufox server --port 9377 &

Smoke test via Hermes:

1hermes chat -q "use the browser to fetch the title of example.com"

Aliases for day-to-day:

1alias camofox-start="python -m camoufox server --port 9377 &"
2alias camofox-stop="pkill -f 'camoufox server'"

What Actually Went Wrong

Ollama native vs compat endpoint. Already covered above, but it cost a couple of hours. If a cloud model hangs at the provider level, check whether your gateway is talking to the right API path.

Camoufox silent failure. The binary fetch is not in postinstall. No error, just no browser. Always run python -m camoufox fetch explicitly before the first start.

Hermes gateway dependency. First start printed nothing useful. Run in foreground before daemonizing so you can see the actual error.

Bifrost and empty API keys. Ollama initially rejected non-empty dummy keys in the Authorization header. Setting the key to an empty string before switching to the real OLLAMA_API_KEY got things unstuck. The order mattered.

Session corruption in OpenCode. An OpenCode session used to orchestrate this setup got a zombie assistant message from an interrupted stream. The message had an empty string for content. Bifrost converted that empty string to null before forwarding, and the upstream provider rejected the entire next request with messages.315.member.content must not be null. Every retry included the same zombie in history, so the error was permanent until manually fixed.

The fix is to delete the zombie row directly from the OpenCode SQLite database:

1sqlite3 ~/.local/share/opencode/opencode.db \
2  "DELETE FROM message WHERE id = '<zombie-message-id>' AND session_id = '<session-id>';"

There is no built-in repair command. Back up the database before editing it.

Final State

Services running when done:

bifrost    :24242  -- Ollama cloud routes
ollama     :11434  -- cloud relay (gemma4, qwen3.5)
camoufox   :9377   -- CDP browser endpoint
hermes     gateway -- Telegram bot (restricted to single user)

The agent receives messages on Telegram, runs tool calls locally (terminal, browser, file ops, web search), and routes all model traffic through Bifrost.

What’s Left

The original plan included running a local Qwen3.5-27B model via llama.cpp with Metal acceleration. That is a 13.5GB download and a build-from-source step, so it was deferred to the next session. The cloud routing setup works fine in the meantime.

References

• Hermes Agent: https://github.com/NousResearch/hermes-agent
• Hermes docs: https://hermes-agent.nousresearch.com/docs/
• Bifrost: https://github.com/maximhq/bifrost
• Camoufox: https://github.com/daijro/camoufox
• OpenCode: https://opencode.ai

The tasks were small. The backlog was not. This is the worst possible combo for a normal workflow because the fixed overhead dominates: branch setup, dependency sync, commits, reviewable diffs, and PR management. Doing it serially feels safe, but it is slow in a way that doesn’t show up in estimates.

So I tried the obvious cheat: run multiple AI agents in parallel.

It worked, but only after I tripped over three problems that are easy to reproduce if you try this without thinking through git state, shells, and sandbox boundaries.

What the system looked like (so you don’t think this is a demo problem)

I ended up writing a small orchestration script with three phases:

1. Create (or reuse) one worktree per task.
2. Install dependencies in parallel.
3. Run codex exec in batched parallel, then push branches and open draft PRs.

The key detail is that batching wasn’t “run everything at once”. It was “parallel within a batch, sequential across batches”, grouped by expected file overlap.

Here is the redacted shape of the script (this is not pseudocode; it’s a sanitized skeleton of what I ran):

 1#!/usr/bin/env zsh
 2set -euo pipefail
 3
 4WORKTREE_DIR=".worktrees"
 5BASE_BRANCH="<base-branch>"
 6LOG_DIR="${WORKTREE_DIR}/_logs"
 7
 8mkdir -p "${WORKTREE_DIR}" "${LOG_DIR}"
 9
10# Phase 1: worktrees + deps (parallel installs)
11hdr "Phase 1: Setting up worktrees"
12git worktree add -b "<feature-branch>" "${WORKTREE_DIR}/<lane>" "${BASE_BRANCH}"
13(cd "${WORKTREE_DIR}/<lane>/frontend" && bun install --frozen-lockfile) &
14(cd "${WORKTREE_DIR}/<lane>/backend"  && uv sync) &
15wait
16
17# Phase 2: batched parallel codex exec (one log per lane)
18hdr "Phase 2: Running Codex"
19codex exec --dangerously-bypass-approvals-and-sandbox -C "${WORKTREE_DIR}/<lane>" -m "${CODEX_MODEL}" "<prompt>" > "${LOG_DIR}/<lane>.log" 2>&1 &
20wait
21
22# Phase 3: push + PRs
23hdr "Phase 3: PRs to ${BASE_BRANCH}"
24git -C "${WORKTREE_DIR}/<lane>" push -u origin "<feature-branch>"
25gh pr create --base "${BASE_BRANCH}" --head "<feature-branch>" --draft --body "<templated body>"

Notice what this gives you immediately: one directory per lane, one log file per lane, and a clean boundary between “make changes” and “publish changes”.

Now for the parts that went wrong.

Failure #1: The shell lied about what “Bash” means

The first failure happened before any parallelism even started. The orchestration was written with associative arrays, and the system shell rejected it with:

1declare: -A: invalid option

This is the kind of bug that looks embarrassing until you realize it is inevitable. macOS ships an older Bash; other environments may default to sh; CI runners may use different interpreters. If your control-plane script can’t build the lane-to-branch mapping reliably, your entire run becomes a dice roll.

The fix was not “make it work on my machine”. The fix was pinning the interpreter and failing fast (set -euo pipefail). Once the script ran under the interpreter it was written for, this class of failure disappeared.

Failure #2: The sandbox was fine until it touched .git

The second failure was more instructive: everything looked successful until commit/push time, and then git failed when trying to write worktree metadata:

1fatal: Unable to create '.git/worktrees/<lane>/index.lock': Operation not permitted

The important mental model is: “can edit files” is not equivalent to “can finish delivery”. Git history mutation needs to create locks and update metadata under .git/. Sandboxed agents often allow workspace file edits while denying .git writes by design.

The fix is to treat permissions as architecture. In my runbook now, implementation and version-control mutation are distinct phases. I keep guardrails on while generating and editing code, and only elevate privileges for commit, rebase, push, and PR creation. That sounds procedural, but it is the difference between parallelism and chaos.

Failure #3: The PR diff was computed against the wrong reality

The third failure was the “ghost diff” class: the PR contained changes that “weren’t part of the task” even though the worktree looked clean locally. This is not a GitHub bug. It is a merge-base bug.

PR diff is computed from:

1merge_base(feature, base) .. feature

If your base ref is stale when the feature branch is compared, merge_base resolves behind your intended base, and commits that should be “already in base” appear as if they were introduced by the feature.

The only reliable approach is to verify ancestry before review and rebase if needed:

1git merge-base --is-ancestor origin/<base-branch> HEAD || echo REBASE_REQUIRED
2git diff --name-only origin/<base-branch>...HEAD

This takes seconds and saves you from burning reviewer attention on diff noise.

What changed once I actually read the Codex CLI docs

My first attempt treated Codex like a generic shell command that prints code. The docs made the control surface explicit, and that changed how I designed the run.

Approval and sandbox mode became part of the plan instead of an afterthought. Configuration precedence became something I could reason about when a lane behaved differently. Non-interactive codex exec became the default because it is scriptable and produces logs you can archive. And built-in review flows (like /review) became a lane-local quality gate before humans ever see the PR.

I also stopped mixing two different kinds of parallelism. Worktrees are for isolating tasks across lanes. Multi-agent mode is for decomposing one task inside a lane. When you treat them as separate tools, your system stays debuggable.

The boring checklist

At this point the process is intentionally boring, which is exactly what you want near a deadline. My checklist is short:

• If the lane can’t commit/push, it doesn’t “count” as done. Fix the permission boundary first.
• If the PR diff scope isn’t what you expect, stop and fix ancestry before asking for review.
• If a lane doesn’t have a log file, assume it didn’t run deterministically.

Appendix: Redacted Script Skeleton

Below is a condensed, anonymized skeleton of the script I used. It preserves the mechanics (worktrees, batching, logs, PR creation) but removes identity details.

 1#!/usr/bin/env zsh
 2set -euo pipefail
 3
 4BASE_BRANCH="<base-branch>"
 5WORKTREE_DIR=".worktrees"
 6LOG_DIR="${WORKTREE_DIR}/_logs"
 7
 8mkdir -p "${WORKTREE_DIR}" "${LOG_DIR}"
 9
10typeset -A TASK_BRANCH TASK_TITLE TASK_PLAN TASK_LABEL
11TASK_BRANCH[001]="feature/lane-001"
12TASK_TITLE[001]="Lane 001: <title>"
13TASK_PLAN[001]="docs/plans/<plan>.md"
14TASK_LABEL[001]="<label>"
15
16BATCH_1=(001 002)
17BATCH_2=(003)
18
19setup_lane() {
20  local id="$1"
21  local branch="${TASK_BRANCH[$id]}"
22  local worktree="${WORKTREE_DIR}/lane-${id}"
23
24  if [[ -d "${worktree}" ]]; then
25    echo "Reusing ${worktree}"
26  else
27    git worktree add -b "${branch}" "${worktree}" "${BASE_BRANCH}"
28  fi
29
30  (cd "${worktree}/frontend" && bun install --frozen-lockfile) &
31  (cd "${worktree}/backend"  && uv sync) &
32  wait
33}
34
35run_lane() {
36  local id="$1"
37  local worktree="${WORKTREE_DIR}/lane-${id}"
38  local log="${LOG_DIR}/lane-${id}.log"
39  local prompt="<redacted prompt built from plan + conventions>"
40
41  codex exec --dangerously-bypass-approvals-and-sandbox -C "${worktree}" "${prompt}" > "${log}" 2>&1
42}
43
44create_pr() {
45  local id="$1"
46  local branch="${TASK_BRANCH[$id]}"
47  local title="${TASK_TITLE[$id]}"
48  local worktree="${WORKTREE_DIR}/lane-${id}"
49
50  git -C "${worktree}" push -u origin "${branch}"
51  gh pr create --base "${BASE_BRANCH}" --head "${branch}" --draft --title "${title}" --body "<templated body>"
52}

References

• Codex docs hub: https://developers.openai.com/codex
• Codex CLI overview: https://developers.openai.com/codex/cli
• Codex CLI features: https://developers.openai.com/codex/cli/features
• Codex multi-agent docs: https://developers.openai.com/codex/multi-agent
• Codex security/sandbox docs: https://developers.openai.com/codex/security
• Codex config basics: https://developers.openai.com/codex/config-basic
• Codex config reference: https://developers.openai.com/codex/config-reference
• Codex CLI GitHub README: https://github.com/openai/codex/blob/main/README.md
• Ralph loop agent (Vercel Labs): https://github.com/vercel-labs/ralph-loop-agent
• Ralph from first principles (Geoffrey Huntley): https://www.youtube.com/watch?v=4Nna09dG_c0
• Git worktree docs: https://git-scm.com/docs/git-worktree