OpenAI Stream Usage Needs a Complete Final Receipt

The reproduced stream completed its HTTP body and ended with data: [DONE], yet it contained zero usage chunks. That is a successful transport result and an unusable billing result.

Accept a streamed request as a usage receipt only when the client requested stream_options.include_usage, exactly one non-null usage object appears in the final JSON event, its choices array is empty, token arithmetic is consistent, and [DONE] closes the stream. Quarantine an interrupted or inconsistent stream instead of quietly recording zero tokens or inventing provider truth.

Put the Metering Boundary After finish_reason

OpenAI chat-completion streaming uses Server-Sent Events (SSE): the response body contains ordered data: frames rather than one final JSON document. Ordinary chunks carry incremental choices data. A choice can reach finish_reason: "stop" before the stream itself has delivered the terminal usage event.

Official OpenAI streaming guidance enables usage with stream_options={"include_usage": True}. The streaming-events reference defines the important edge: ordinary chunks have null usage, the extra final chunk has an empty choices array and populated usage, and an interrupted or cancelled stream may never deliver that last chunk.

Stopping iteration at the first non-null finish_reason therefore discards the event required for exact server-reported metering. Indexing choices[0] on every event creates a different bug because the usage event deliberately has no choice. The client must model content completion and receipt completion as separate states.

A gateway adds another ownership layer. The existing LiteLLM credential-boundary guide explains provider keys, virtual keys and spend ownership; the test below focuses on the narrower contract that a single stream must satisfy before any gateway ledger accepts it.

Define acceptance before writing the parser

For a request that requires provider-reported usage, promote the stream only when all of these assertions are true:

1. The request explicitly asks for streamed usage.
2. JSON events remain parseable and occur before the terminal marker.
3. Exactly one event contains a non-null usage object.
4. That event is the final JSON event and has choices: [].
5. prompt_tokens + completion_tokens == total_tokens for the fields the endpoint returned.
6. A single [DONE] marker appears after the usage event.

HTTP 200, readable text and finish_reason remain useful transport or content signals. None is a substitute for this accounting receipt.

Build an Owned Loopback SSE Workbench

This lab uses Python’s standard library and curl on loopback. It spends no provider tokens, stores no API key and opens no public listener. Its synthetic counts prove parser behavior, not the token count of a real model request.

Refuse stale state and identify the tools

Run every tested block in one fresh Bash session. The first block refuses an existing path or occupied port, creates a mode-0700 workspace and records the exact owner marker used by cleanup.

set -Eeuo pipefail
lab_root=/tmp/voxfor-openai-stream-usage-167
port=18767
owner_token=voxfor-openai-stream-usage-167
server_pid=''

test ! -e "$lab_root"
for tool in python3 curl; do command -v "$tool" >/dev/null; done
python3 - "$port" <<'PY'
import socket, sys
s = socket.socket()
try:
    s.bind(("127.0.0.1", int(sys.argv[1])))
finally:
    s.close()
PY
install -d -m 0700 "$lab_root"
printf '%s\n' "$owner_token" >"$lab_root/.owner-marker"
printf 'owned_path=%s python=%s curl=%s port=%s\n' \
  "$lab_root" "$(python3 --version 2>&1)" "$(curl --version | head -n1)" "$port"

Loopback ownership is an accident-prevention boundary, not isolation for untrusted code. Keep production prompts, responses and credentials out of a parser test. If a real compatibility probe is required later, use a synthetic prompt and a dedicated low-privilege key.

Serve complete, missing and inconsistent receipts

Our fixture always emits three ordinary chunks, including one finish_reason: "stop". It appends usage only when the request asks for it. Two headers create negative controls: truncated omits both the usage event and [DONE], while bad-total reports 12 tokens although the two components total 11.

cat >"$lab_root/server.py" <<'PY'
import json, sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

def event(choices, usage_marker=False, usage=None):
    value = {"id":"chatcmpl-voxfor-167","object":"chat.completion.chunk",
             "created":1786646400,"model":"openai-compatible-lab","choices":choices}
    if usage_marker:
        value["usage"] = usage
    return value

class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"
    def log_message(self, fmt, *args):
        return
    def do_POST(self):
        if self.path != "/v1/chat/completions":
            self.send_error(404); return
        request = json.loads(self.rfile.read(int(self.headers.get("content-length", "0"))))
        if request.get("stream") is not True:
            self.send_error(400, "stream must be true"); return
        include = bool(request.get("stream_options", {}).get("include_usage"))
        scenario = self.headers.get("X-Lab-Scenario", "complete")
        frames = [
          event([{"index":0,"delta":{"role":"assistant","content":"metered "},"finish_reason":None}], include, None),
          event([{"index":0,"delta":{"content":"stream"},"finish_reason":None}], include, None),
          event([{"index":0,"delta":{},"finish_reason":"stop"}], include, None),
        ]
        if include and scenario != "truncated":
            total = 12 if scenario == "bad-total" else 11
            frames.append(event([], True, {"prompt_tokens":7,"completion_tokens":4,"total_tokens":total}))
        body = "".join(f"data: {json.dumps(x,separators=(',',':'))}\n\n" for x in frames)
        if scenario != "truncated":
            body += "data: [DONE]\n\n"
        raw = body.encode()
        self.send_response(200)
        self.send_header("Content-Type", "text/event-stream")
        self.send_header("Content-Length", str(len(raw)))
        self.end_headers()
        self.wfile.write(raw); self.wfile.flush()

ThreadingHTTPServer(("127.0.0.1", int(sys.argv[1])), Handler).serve_forever()
PY
chmod 700 "$lab_root/server.py"
python3 "$lab_root/server.py" "$port" >"$lab_root/server.log" 2>&1 &
server_pid=$!
printf '%s\n' "$server_pid" >"$lab_root/server.pid"
for _ in {1..50}; do
  if (exec 3<>"/dev/tcp/127.0.0.1/$port") 2>/dev/null; then
    exec 3>&-
    break
  fi
  sleep 0.05
done
kill -0 "$server_pid"

Audit event order, shape and arithmetic

Receipt auditing ignores blank SSE lines, parses only data: frames and treats [DONE] as a terminal marker rather than JSON. When usage is required, zero or multiple usage chunks are failures. Event order, empty choices and arithmetic are independent checks, so a gateway cannot satisfy the contract by injecting a plausible-looking object in the wrong place.

cat >"$lab_root/audit_stream.py" <<'PY'
import argparse, json
from pathlib import Path
p = argparse.ArgumentParser()
p.add_argument("stream_file", type=Path)
p.add_argument("--require-usage", action="store_true")
a = p.parse_args()

events, done_count = [], 0
for raw in a.stream_file.read_text().splitlines():
    if not raw.startswith("data:"):
        continue
    payload = raw[5:].strip()
    if payload == "[DONE]":
        done_count += 1
        if done_count > 1:
            raise SystemExit("REJECT duplicate_done")
        continue
    if done_count:
        raise SystemExit("REJECT event_after_done")
    events.append(json.loads(payload))
if done_count != 1:
    raise SystemExit("REJECT missing_done")
if not events:
    raise SystemExit("REJECT no_json_events")

usage_events = [(i,e) for i,e in enumerate(events) if e.get("usage") is not None]
if a.require_usage and len(usage_events) != 1:
    raise SystemExit(f"REJECT usage_chunk_count={len(usage_events)}")
if not usage_events:
    print(f"TRANSPORT_COMPLETE json_events={len(events)} usage_chunks=0 done=yes")
    raise SystemExit(0)

i, terminal = usage_events[0]
usage = terminal["usage"]
if i != len(events)-1:
    raise SystemExit("REJECT usage_not_final_json_event")
if terminal.get("choices") != []:
    raise SystemExit("REJECT usage_choices_not_empty")
expected = usage["prompt_tokens"] + usage["completion_tokens"]
if usage["total_tokens"] != expected:
    raise SystemExit(f"REJECT total_mismatch expected={expected} observed={usage['total_tokens']}")
print(f"ACCEPT json_events={len(events)} usage_chunks=1 choices_empty=yes "
      f"prompt={usage['prompt_tokens']} completion={usage['completion_tokens']} "
      f"total={usage['total_tokens']} done=yes")
PY
chmod 700 "$lab_root/audit_stream.py"

Compare Transport Completion With Receipt Completion

Two matched controls send the same model and message to the same endpoint. Only the request contract changes. Keeping everything else identical makes the missing receipt attributable to include_usage, not prompt content or transport behavior.

Prove that [DONE] can arrive without usage

First, the baseline request omits stream_options. Its ordinary audit reports transport completion. The metering audit must reject it with usage_chunk_count=0.

request_no_usage='{"model":"openai-compatible-lab","messages":[{"role":"user","content":"receipt"}],"stream":true}'
curl --fail --silent --show-error --no-buffer \
  -H 'Content-Type: application/json' -d "$request_no_usage" \
  "http://127.0.0.1:$port/v1/chat/completions" >"$lab_root/no-usage.sse"
python3 "$lab_root/audit_stream.py" "$lab_root/no-usage.sse"
if python3 "$lab_root/audit_stream.py" --require-usage "$lab_root/no-usage.sse" \
     >"$lab_root/no-usage.out" 2>"$lab_root/no-usage.err"; then
  exit 21
fi
grep -q 'REJECT usage_chunk_count=0' "$lab_root/no-usage.err"

This result prevents a common accounting default: zero is a measured value, while missing usage is unknown. Store the latter as incomplete or quarantined. Turning absence into zero systematically under-bills interrupted or misconfigured streams and destroys the evidence needed for reconciliation.

Accept one terminal usage event

Now add stream_options.include_usage: true. The fixture places usage after the choice with finish_reason: "stop" and immediately before [DONE]. A consumer must keep reading even though content generation already finished.

request_with_usage='{"model":"openai-compatible-lab","messages":[{"role":"user","content":"receipt"}],"stream":true,"stream_options":{"include_usage":true}}'
curl --fail --silent --show-error --no-buffer \
  -H 'Content-Type: application/json' -d "$request_with_usage" \
  "http://127.0.0.1:$port/v1/chat/completions" >"$lab_root/complete.sse"
python3 "$lab_root/audit_stream.py" --require-usage "$lab_root/complete.sse" \
  | tee "$lab_root/complete.result"

LiteLLM’s usage documentation describes the same compatible shape: an additional chunk before [DONE], empty choices and populated usage. Langfuse’s OpenAI integration guards choice access when reading that final event. Those examples are useful compatibility evidence, but each deployed gateway still needs its own probe because versions, routes and custom base URLs can differ.

Reject Incomplete or Internally Inconsistent Streams

A TCP close is not proof that every intended SSE event arrived. Proxies, clients and upstreams can terminate a response after visible text is complete but before usage. A real 499 investigation should follow the request path described in the NGINX client-closed-request workflow; the accounting parser should still fail closed regardless of which hop owns the disconnect.

Exercise truncation and bad arithmetic separately

One negative scenario contains the finish_reason event but omits usage and [DONE]. The second completes transport and includes usage, yet reports an inconsistent total. They are separate faults with separate remediation owners, not two aliases for a generic error.

curl --fail --silent --show-error --no-buffer \
  -H 'Content-Type: application/json' -H 'X-Lab-Scenario: truncated' \
  -d "$request_with_usage" "http://127.0.0.1:$port/v1/chat/completions" \
  >"$lab_root/truncated.sse"
if python3 "$lab_root/audit_stream.py" --require-usage "$lab_root/truncated.sse" \
     >"$lab_root/truncated.out" 2>"$lab_root/truncated.err"; then exit 22; fi
grep -q 'REJECT missing_done' "$lab_root/truncated.err"

curl --fail --silent --show-error --no-buffer \
  -H 'Content-Type: application/json' -H 'X-Lab-Scenario: bad-total' \
  -d "$request_with_usage" "http://127.0.0.1:$port/v1/chat/completions" \
  >"$lab_root/bad-total.sse"
if python3 "$lab_root/audit_stream.py" --require-usage "$lab_root/bad-total.sse" \
     >"$lab_root/bad-total.out" 2>"$lab_root/bad-total.err"; then exit 23; fi
grep -q 'REJECT total_mismatch expected=11 observed=12' "$lab_root/bad-total.err"
printf 'negative_no_usage=%s negative_truncated=%s negative_bad_total=%s\n' \
  "$(cat "$lab_root/no-usage.err")" \
  "$(cat "$lab_root/truncated.err")" \
  "$(cat "$lab_root/bad-total.err")"
printf 'receipt_complete=yes request_include_usage=yes usage_final=yes choices_empty=yes arithmetic=yes done=yes\n'

Here is the representative output from the full run. Counts are synthetic and deterministic; the pass/fail contract is the transferable result.

owned_path=/tmp/voxfor-openai-stream-usage-167 python=Python 3.13.5 curl=curl 8.14.1 port=18767
TRANSPORT_COMPLETE json_events=3 usage_chunks=0 done=yes
ACCEPT json_events=4 usage_chunks=1 choices_empty=yes prompt=7 completion=4 total=11 done=yes
negative_no_usage=REJECT usage_chunk_count=0 negative_truncated=REJECT missing_done negative_bad_total=REJECT total_mismatch expected=11 observed=12
receipt_complete=yes request_include_usage=yes usage_final=yes choices_empty=yes arithmetic=yes done=yes
cleanup_scope=/tmp/voxfor-openai-stream-usage-167 absent=yes

Reconcile unknown receipts without double counting

Give every outbound request a stable application receipt ID before opening the stream. Store provider request identifiers, gateway identity, model, request start, terminal state and usage fields under that ID. When a stream is incomplete, preserve the partial record as usage_unknown; do not retry the charge insert under a new identity.

Business-action retries need the same discipline. The AI action idempotency workflow shows why a transport retry and a new business effect are different decisions. Metering should upsert one request receipt, while the application separately decides whether it is safe to repeat generation or a downstream tool action.

A provider dashboard or later reconciliation export may resolve an unknown receipt. Record the source and reconciliation time instead of overwriting history without provenance. Local tokenization can support an explicitly labeled estimate, but tokenizer revision, message framing, tool schemas, cached tokens and provider-specific accounting mean an estimate is not the same object as provider-reported usage.

Carry the Receipt Contract Into Production

Probe every SDK, proxy route and OpenAI-compatible endpoint you operate. LangChain’s stream_usage reference notes that custom base URLs may not support the behavior. A compatibility label is not enough to infer terminal usage semantics.

For each route, capture a secret-free synthetic request and require the same invariants as the lab. Record gateway and SDK versions, model alias, whether include_usage reached upstream, whether exactly one empty-choice usage event arrived, whether [DONE] followed, and how cancellation is represented. Repeat the probe after upgrades or routing changes.

Teams hosting an application gateway on VPS infrastructure with root-level control can keep its raw listener on loopback and put a reviewed HTTPS proxy in front. Hosting control does not create usage integrity by itself; the parser and reconciliation policy remain application-owned. Apply the Caddy validate-and-reload sequence when proxy timeouts or streaming headers change, then rerun both complete and truncated controls.

Receipt delivery downstream also has a failure budget. If usage records pass through an observability collector, a full queue can separate accepted model calls from exported telemetry. The OpenTelemetry Collector outage-buffer test provides the next durability check. Prefer a durable accounting store as the billable source of truth and treat dashboards as projections unless their delivery guarantees are explicitly proven.

Accept the lab receipt only when the fixture binds to 127.0.0.1:18767; the baseline ends at [DONE] but fails usage-required validation; the positive request yields four JSON events with exactly one final empty-choice usage object; prompt 7 plus completion 4 equals total 11; truncation fails for missing [DONE]; the inconsistent total fails with expected 11 and observed 12; and marker-scoped cleanup leaves no fixture. In production, rerun the same assertions against every reviewed gateway route with synthetic data and retain versioned evidence.

Remove only the marked fixture

Stop the exact recorded child, verify the owner token and delete only files inside the marker-owned directory. Never use a wildcard, kill an unverified PID or reuse this cleanup against a service path.

if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
  kill "$server_pid"
  wait "$server_pid" 2>/dev/null || true
fi
test "$(cat "$lab_root/.owner-marker")" = "$owner_token"
find "$lab_root" -mindepth 1 -maxdepth 1 -type f -delete
rmdir "$lab_root"
test ! -e "$lab_root"
printf 'cleanup_scope=%s absent=yes\n' "$lab_root"

When a receipt assertion fails, do not turn the missing value into zero and do not promote the record to billable truth. Preserve the secret-free SSE, parser error, request receipt ID, gateway version and proxy status; keep the last accepted parser and route configuration unchanged; fix only the owning SDK, proxy, gateway or reconciliation path; and rerun the complete plus negative controls. A production rollback restores the previous reviewed application and proxy configuration through their normal change owner, while unresolved requests remain quarantined for provider-side reconciliation.

Keep Transport Health Separate From Billable Truth

A readable response can finish while its usage receipt is absent. A terminal usage object can arrive while its arithmetic is inconsistent. A dashboard can miss an accepted record after the model call has already cost money. Each state has a different owner.

Use provider-reported usage when the complete receipt passes, preserve local estimates under an explicit estimate label, and quarantine missing or inconsistent receipts under stable request identities. Transport health tells you whether bytes moved; a complete final receipt tells you whether this stream is ready to enter the ledger.