14 — Cost Model & Budget Governance

Purpose. This document defines where LLM/embedding costs arise in the Knoll Analyzer × GRAG integration, provides a per-analysis cost-estimation worksheet at the planning scale (151 questions per ADR-010, 46 checklist positions, 11 success levers), specifies the cost-attribution architecture (X-Pipeline-Id → ledger), the budget-governance mechanics (keyvault monthly_budget_usd / rate_limit_rpm, hard 402/429), the 80 %-alert job, the cost-control levers, and the monthly firm reporting recipe. It implements plan 1.6, 7.3 and the budget/model parts of 6.6.

Status / verified against: 2026-07-06 · repo document-processing-pipelines @ db63a95 · plan tasks/todo.md rev. 2 (adversarial-review corrected). Ledger paths, keyvault fields, attribution headers and cache semantics verified in source (ledger/src/ledger/api.py, ai-gateway/src/ai_gateway/keyvault/*, orchestrator/src/orchestrator/worker.py:385).

Sibling docs: architecture in 01-architecture.md, request patterns in 02-grag-api-cookbook.md, run-id conventions in 03-id-conventions.md, keyvault provisioning in 04-provisioning-runbook.md, ai_runs schema in 06-knoll-db-schema.md, pipeline steps in 08-gutachten-pipeline-spec.md, chat lane in 09-chat-integration.md, ingest lane in 10-document-pipeline.md, 402/429 handling in 11-resilience-and-errors.md, attribution gaps in 13-platform-gaps-issues.md.

1. Where costs arise, per feature

The cost ledger records every paid call. Two row classes matter:

• USD-bearing rows — ai-gateway LLM/embed calls (LiteLLM computes usd + tokens). These are the only rows that consume monthly_budget_usd.
• usd=0 / units-only rows — rerank-fusion /rerank+/retrieve and every groundedness scoring call emit units=1, usd=0; docfold/scraping/enrichment record units only (pages, requests, characters). They appear in the ledger for observability but cost nothing at the gateway (docfold engines are local: Docling, PaddleOCR etc.).

⚠ Rule of thumb: completions dominate. Embeddings for a full Analyse corpus cost cents; the Gutachten pipeline and chat turns are where the Euro-relevant spend is.

2. Per-Analyse cost-estimation worksheet

2.1 Unit-price variables — resolve at runtime, never hardcode

TODO-VERIFY: pull current prices from the model catalog at runtime and fill this table per environment. Do not copy prices from memory or vendor websites — the catalog is what LiteLLM actually bills against.

curl -s "$GRAG_URL/ai-gateway/api/v1/models?provider=openai" \
  -H "Authorization: Bearer $GRAG_API_KEY" \
  -H "X-Tenant-ID: $GRAG_TENANT" \
  | jq '.models[] | select(.model | test("gpt-4o-mini|text-embedding-3-small"))
        | {model, pricing, max_input_tokens, max_output_tokens}'

Each entry exposes pricing.input_cost_per_token and pricing.output_cost_per_token (USD per single token; multiply by 1e6 for the per-1M figure).

2.2 Scale constants (canonical / measured) and planning assumptions

Assumptions (tune after the 8.3 pilot measures real values — these are planning inputs, not platform facts):

2.3 Formula rows

A. Ingest (embeddings, model = embedding model):

B. Chat, per conversation (completion model m):

C. Gutachten-Pipeline, per run (completion model(s) per step; fallback retries bill again on the fallback model):

D. Per-Analyse total:

Cost(Analyse) ≈ Σ_models [ T_in(m) × P_in(m)/1M + T_out(m) × P_out(m)/1M ]
             + T_emb × P_emb/1M

Token budget with defaults (1 pipeline run + 3 chat conversations):
  T_in  ≈ 241k + 3×192k ≈ 0.82 M prompt tokens
  T_out ≈  42k + 3×8k   ≈ 0.07 M completion tokens
  T_emb ≈ 0.48 M embedding tokens

Multiply by live prices from 2.1. Budget at least 2× this figure for the pilot: Partner-review gates between steps (5.3) cause step re-runs, prompt-JSON validation retries (1.14) re-bill, and knoll_ai_settings.detail_level can raise max_tokens on step 5. Record actuals per run (Section 3) and replace these estimates after 8.3.

TODO-VERIFY: whether the compound search/rerank on a dense collection emits a paid query-embedding ledger row per call (query-side embedding path for dense collections is not explicit in source) — check GET /ledger/api/v1/ledger?since=<t> after a dev chat turn; if yes, add T × ~30 tok × P_emb per conversation (negligible but should be known).

3. Attribution architecture: per-run → per-Analyse cost

3.1 The X-Pipeline-Id convention

The ledger keys per-run attribution on the pipeline_id column, populated from the X-Pipeline-Id request header (read by ai-gateway routes/completion.py:78,122, routes/responses.py:111, and by pipeline_common.ledger.record_cost). Per 03-id-conventions.md §8, Knoll stamps one X-Pipeline-Id per ai_runs row, format knoll-<scope>-<step>-<runid> (e.g. knoll-4b0c77de-scoring-4e21ac03), stored in ai_runs.pipeline_id at run start.

Every direct gateway call the Knoll backend makes (Gutachten steps, checklist classification, lead scoring, Begründungstexte, Projektsteckbriefe) carries it:

curl -s -X POST "$GRAG_URL/ai-gateway/api/v1/chat/completions" \
  -H "Authorization: Bearer $GRAG_API_KEY" \
  -H "X-Tenant-ID: $GRAG_TENANT" \
  -H "X-Pipeline-Id: knoll-4b0c77de-scoring-4e21ac03" \
  -H "Content-Type: application/json" \
  -d '{
        "model": "openai/gpt-4o-mini",
        "messages": [{"role": "user", "content": "…"}],
        "max_tokens": 1500,
        "fallback": ["openai/gpt-4o"]
      }'

grag-client (2.5) must inject the header automatically whenever a call runs inside an ai_runs context — a missing header means the spend lands unattributed (tenant-level only).

3.2 Which lanes carry attribution — and which do not

3.3 The ⚠ correct ledger endpoints

External base = https://app.grag.ai ($GRAG_URL); Traefik prefix /ledger + internal path /api/v1/ledger/... — note the doubled ledger segment. All reads are scoped to X-Tenant-ID (RLS); cross-tenant queries need ?tenant= plus an admin key in LEDGER_ADMIN_API_KEYS (403 otherwise).

Per-run totals (one ai_runs row = one pipeline id; per-Analyse = sum over the Analyse's ids, 3.4):

curl -s "$GRAG_URL/ledger/api/v1/ledger/totals?pipeline_id=knoll-4b0c77de-scoring-4e21ac03" \
  -H "Authorization: Bearer $GRAG_API_KEY" \
  -H "X-Tenant-ID: $GRAG_TENANT"

{
  "pipeline_id": "knoll-4b0c77de-scoring-4e21ac03",
  "calls": 1,
  "failed_calls": 0,
  "total_tokens": 13200,
  "total_cost_usd": 0.0,
  "max_latency_ms": 0,
  "avg_latency_ms": 0
}

(Field names are load-bearing — wire-compatible with the orchestrator budget poller; ledger/models.py:49.)

Tenant-level spend for the running month:

MONTH_START=$(date -d "$(date +%Y-%m-01)" +%s)
curl -s "$GRAG_URL/ledger/api/v1/ledger/spend?group_by=service&since=$MONTH_START" \
  -H "Authorization: Bearer $GRAG_API_KEY" \
  -H "X-Tenant-ID: $GRAG_TENANT"

{
  "tenant": "knoll",
  "group_by": "service",
  "since": 1751760000,
  "until": null,
  "rows": [
    {"dimension": "ai-gateway", "calls": 812, "units": 2100000, "usd": 3.42, "failed_calls": 2}
  ]
}

TODO-VERIFY: the exact service dimension values that appear in the live rollup (expected: producer service names like ai-gateway, docfold, mediafold, enrichment, scraping) — run the spend query on dev after a full ingest+chat+pipeline cycle and record the observed set here.

Note: ai-gateway also serves a local mirror at GET /ai-gateway/api/v1/ledger + /ledger/totals (its anomaly/percentile source of truth). Knoll uses the ledger service endpoints above — the mirror covers only ai-gateway's own rows.

3.4 How Knoll aggregates per-analysis cost

Knoll DB (06-knoll-db-schema.md) is the source of truth for the roll-up; the ledger is the metering source:

1. Every LLM-consuming unit of work is an ai_runs row (2.1: analysis_id nullable, agent_id, step, status, model, tokens, cost, …).
2. On terminal status (succeeded/failed/cancelled per 06 §2.2 ai_run_status — budget-402 runs land as failed + error='budget_exceeded', 08 §7.1), the Knoll backend calls /ledger/totals?pipeline_id=<ai_runs.pipeline_id> (03 §8 format) once and writes total_cost_usd → ai_runs.cost, total_tokens → ai_runs.tokens. (Ledger writes are async via Redis streams — poll once ~30 s after terminal state, or accept eventual consistency and let the nightly job correct it.)
3. Ingest jobs are attributed via their orchestrator jobId from grag_refs (3.2) and rolled into the owning analysis the same way (either as an ai_runs row of type ingest or a files-level cost column — pick one in 2.1 implementation).
4. Per-analysis cost = Σ ai_runs.cost where analysis_id = :id (+ its ingest jobs). Cached, never recomputed from GRAG on page load — the 6.4 pipeline panel and 6.6 budget tab read Knoll SQL.
5. Nightly reconcile job: re-fetch /ledger/totals for runs updated in the last 48 h and correct drift (late ledger rows, fallback retries).

4. Budget governance

4.1 Keyvault semantics (plan 1.6)

The Kanzlei's provider keys are stored per tenant in the ai-gateway keyvault (BYOK, ADR 0029; GATEWAY_KEYVAULT_ENABLED=true in the prod env example). Governance fields on each stored key: enabled, monthly_budget_usd, rate_limit_rpm, expires_at (keyvault/models.py:150-160).

curl -s -X POST "$GRAG_URL/ai-gateway/api/v1/keys" \
  -H "Authorization: Bearer $GRAG_API_KEY" \
  -H "X-Tenant-ID: $GRAG_TENANT" \
  -H "Content-Type: application/json" \
  -d '{
        "provider": "openai",
        "secret": "sk-…",
        "label": "kanzlei-knoll-openai",
        "allowed_services": ["ai-gateway"],
        "scope_level": "tenant",
        "monthly_budget_usd": 200,
        "rate_limit_rpm": 60
      }'

Verified allowed_services vocabulary (ai-gateway/src/ai_gateway/keyvault/models.py:28, SERVICES; enforced by the create route's validator in api/schemas/keys.py): ai-gateway | docfold | scraping | mediafold | enrichment — service names, not capability names; an empty list [] means "any service". Values like "completion" / "embedding" are rejected with 422. For an LLM-provider key, ["ai-gateway"] (or []) is correct — all completion/embedding traffic bills through ai-gateway.

Enforcement semantics (verified keyvault/resolver.py, usage.py):

Fine print an operator must know:

• The budget window is the UTC calendar month (%Y%m, usage.py:39) — not a rolling 30 days. Spend counters live in Redis (DB 5) and are fail-open: a Redis outage means requests pass and the month counter under-counts (budget can overshoot during the outage).
• Governance applies only when a stored tenant key resolves for the provider. A per-request api_key in the body bypasses the vault; no stored key at all → the platform env key is used with no budget cap. Consequence: for the budget to be real, Knoll must store the Kanzlei keys in the vault and never pass per-request keys. grag-client should refuse the api_key body field.
• ⚠ D5 caveat: BFF-lane chat traffic (D3) is billed to the tenant like everything else (the keyvault resolves on X-Tenant-ID, which the BFF forwards), so the budget cap does cover chat — only attribution is missing there (3.2), not governance.

4.2 Budget-exhausted mid-pipeline: designed UX (consumes 2.1 ai_runs, 2.7 email)

A 402 can land mid-Gutachten-run (e.g. after step 3 of 6). Required behavior (spec for 08-gutachten-pipeline-spec.md state machine and 11-resilience-and-errors.md):

1. Never blind-retry a 402/429. 429 → honor Retry-After with bounded backoff. 402 → terminal for the current attempt.
2. On 402 the state machine sets ai_runs.status = 'failed' with error = 'budget_exceeded' (per 08 §7.1; 06 §2.2's ai_run_status enum has no dedicated pause state — "pausiert" is presentation-level only). Keep step, keep all completed step outputs — they are persisted in Knoll DB per step, so nothing is lost.
3. UI: pipeline panel (6.4) shows „Budget erschöpft — Lauf pausiert" with the month-to-date figure; chat UIs disable input with a German banner („Das monatliche KI-Budget der Kanzlei ist ausgeschöpft.").
4. Notification: transactional email (2.7) to all Partner: Analyse, step reached, month-to-date spend, and the two remedies — raise monthly_budget_usd (PATCH /ai-gateway/api/v1/keys/{id}) or wait for the month rollover.
5. Resume re-enters at the failed step (new ai_runs row, new pipeline id per 03 §8). Re-running an identical prior step is cheap anyway: ai-gateway's content-keyed cache (GATEWAY_CACHE_ENABLED, key = canonical body hash) and the orchestrator's per-node cache (1 h TTL, (service_type, payload)-keyed) short-circuit byte-identical calls — but do not rely on cache for correctness, only persistence of step outputs guarantees no re-spend.

4.3 80 %-alert job (Knoll-side cron)

There is no platform-side budget webhook — the gateway only rejects at 100 %. Knoll runs its own threshold alert:

Job: budget-alert            Schedule: hourly
For each stored key in GET /ai-gateway/api/v1/keys (X-Tenant-ID: $GRAG_TENANT):
    pct = key.usage.usd / key.monthly_budget_usd        # usage = month-to-date snapshot
    if pct >= 0.8 and no alert row (key_id, usage.month, "80") exists:
        insert alert row; email Partner + Ops ("80 % des KI-Budgets erreicht")
    if pct >= 0.95 and no alert row (key_id, usage.month, "95") exists:
        insert alert row; email Partner ("95 % — Läufe pausieren in Kürze")
Cross-check (belt & braces): /ledger/api/v1/ledger/spend?since=<month-start>
    summed over rows should ≈ Σ key.usage.usd; alert Ops on >10 % divergence
    (Redis usage counters are fail-open and can under-count).

GET /ai-gateway/api/v1/keys returns each key with a usage snapshot {month, usd, requests} (keyvault/models.py:141) — that is the primary source; the ledger cross-check catches counter drift. Dedupe key (key_id, month, threshold) lives in a small Knoll table; alerts respect benachrichtigungs_praeferenzen (2.1) except the 95 % Partner alert, which is mandatory.

5. Cost-control levers

6. Reporting

6.1 Monthly firm cost report (recipe)

Once per month (1st, 02:00, for the previous month), the Knoll backend materializes a report row set:

SINCE=$(date -d "$(date -d 'last month' +%Y-%m-01)" +%s)
UNTIL=$(date -d "$(date +%Y-%m-01)" +%s)

# 1) Tenant spend by service (what kind of work cost money)
curl -s "$GRAG_URL/ledger/api/v1/ledger/spend?group_by=service&since=$SINCE&until=$UNTIL" \
  -H "Authorization: Bearer $GRAG_API_KEY" -H "X-Tenant-ID: $GRAG_TENANT"

# 2) Tenant spend by provider (which vendor invoice it maps to)
curl -s "$GRAG_URL/ledger/api/v1/ledger/spend?group_by=provider&since=$SINCE&until=$UNTIL" \
  -H "Authorization: Bearer $GRAG_API_KEY" -H "X-Tenant-ID: $GRAG_TENANT"

Per-analysis and per-client breakdown comes from Knoll SQL (Section 3.4), not from the ledger:

-- Cost per analysis / client, previous month
SELECT m.company_name     AS client,
       a.id                AS analysis_id,
       a.package,
       COUNT(r.id)         AS runs,
       SUM(r.tokens)       AS tokens,
       SUM(r.cost)         AS cost_usd
FROM   ai_runs r
JOIN   analyses a  ON a.id = r.analysis_id
JOIN   clients   m ON m.id = a.client_id
WHERE  r.finished_at >= date_trunc('month', now() - interval '1 month')
AND    r.finished_at <  date_trunc('month', now())
GROUP  BY m.company_name, a.id, a.package
ORDER  BY cost_usd DESC;

-- Unattributed remainder (chat lane + spill):
--   tenant_spend_usd (recipe above)  −  SUM(ai_runs.cost)

The unattributed remainder is expected to be mostly BFF chat (3.2). Report it as its own line ("Chat / unattributed") — if it exceeds ~30 % of total spend, prioritize the attribution-header addition to the 1.9 contract work.

Sanity checks in the job: failed_calls ratio per service (>2 % → investigate with GET /ledger/api/v1/ledger?service=…), and the keyvault vs. ledger divergence check from 4.3.

6.2 What lands in the 6.6 Einstellungen → KI tab

All GRAG reads on this page go through the Knoll backend (server-side, with $GRAG_API_KEY + X-Tenant-ID) — the key never reaches the browser (7.7).