09 — Chat Integration Spec (Akquise, KIU, "Frag die Akte")

Purpose. This document specifies how every conversational feature of the Knoll Analyzer is built on the single GRAG chat lane decided in plan 0.3 (D3): the frontend-next BFF POST /next/api/chat/stream. It covers the three chat surfaces (plan 5.1a, 5.1b, 5.2), the conversation lifecycle against the workspaces service, the full SSE wire contract, a client implementation guide, rendering rules for sources/groundedness/confidence, the Akquise lead-scoring post-step, and the security constraints. The Gutachten pipeline (plan 5.3) deliberately does not use this lane — see 08-gutachten-pipeline-spec.md.

Status / verified against: 2026-07-06, repo document-processing-pipelines @ db63a95. SSE contract read from frontend-next/apps/web/app/api/chat/stream/route.ts (1,898 lines; spec-pinned by route.spec.ts); conversation API from the workspaces routers; corrections from the adversarial plan review applied (unauthenticated BFF, no conversation auto-create, no gateway structured output).

Related docs: 01-architecture.md (lanes), 02-grag-api-cookbook.md (headers, error taxonomy), 03-id-conventions.md (DNS-label ids), 06-knoll-db-schema.md (leads, ai_runs, grag_refs), 08-gutachten-pipeline-spec.md, 10-document-pipeline.md (ingest), 11-resilience-and-errors.md, 12-gdpr-compliance.md (conversation deletion), 13-platform-gaps-issues.md (gaps #1 and #5), 14-cost-model.md (X-Pipeline-Id).

0. Security box — read first

⚠ BLOCKER (plan 1.8, platform gap #1): https://app.grag.ai/next/api/* is today publicly reachable with NO authentication. There is no middleware.ts in frontend-next, the Traefik router for /next attaches no auth middleware, and every BFF route injects the platform PIPELINE_API_KEY server-side while trusting the inbound X-Tenant-ID header. Anyone on the internet can chat against a tenant's documents and burn its LLM budget. Consequences for this spec:

1. No real Mandanten data flows through any chat surface until plan 1.8 lands and the plan 0.10a probe returns 401 for unauthenticated calls. Use tenant knoll-dev with fixture data only.
2. All examples below already send Authorization: Bearer $GRAG_API_KEY even though the BFF currently ignores it — so nothing breaks the day 1.8 turns auth on.
3. The browser never talks to GRAG. Every chat call goes browser → Knoll backend route handler (Auth.js session + Knoll role check) → GRAG BFF. The Knoll backend sets X-Tenant-ID from server config (plan 2.8); the tenant header and the GRAG key must never be controllable from, or visible to, the client bundle (plan 7.7). Exposing tenant-header control to the browser reproduces the platform vulnerability inside Knoll.
4. BFF-lane traffic never presents Knoll's own key today (the BFF injects the platform key upstream), so GATEWAY_CALLERS attribution does not cover chat until 1.8 adds a bearer check in the BFF routes (plan 0.5 / D5 caveat).

1. The three chat surfaces

All three consume the same BFF contract (POST /next/api/chat/stream, §3) and get history windowing, query condensation, retrieval, full-text expansion, groundedness, and message persistence for free. Do not fork the ~1.9k-line answer path (plan 0.3). What differs is only the parameterisation:

Conversation ids follow the normative scheme in 03-id-conventions.md §3 — conv-<purpose>-<shortid> with purpose ∈ acquisition|iu|file and <shortid> = the 8-hex shortid of the Knoll-side conversation row. The analysis linkage is not encoded in the id; it lives exclusively in grag_refs (invariant I2: ids are never parsed for meaning).

The prototype's /ai-agents/[id] demo dialog (apps/web/lib/mock-data/agents.ts:154-221) is the 5.1b feature, not the acquisition chat: a client-scoped steering conversation (industry assessment, contradictions/gaps from the questionnaire return, project proposals) about Hartmann Maschinenbau (m-01, analysis a-02). Building only 5.1a would leave the actual prototype feature unimplemented (review finding [9]).

1.1 KIU system-prompt enrichment (5.1b)

The Knoll backend composes the system_prompt server-side per turn from Knoll DB (source of truth — plan 2.1):

1. KIU persona + quality standards (static text, versioned in the ai_agents registry config).
2. Client master data (clients, contacts): company, industry, region, employees, revenue.
3. Questionnaire digest: structured summary of submitted questionnaire_answers for the analysis (chapter-level; not all 151 raw answers).
4. Current expert_reports.lever_scores incl. traffic_light, if an expert-report version exists.
5. Deterministic-cap notes that apply (e.g. Frage 1-8 = Nein → "MarketPosition methodisch capped at 1,5").

Keep the enrichment ≤ ~8,000 characters: the BFF appends its own source-reading guide plus the retrieved sources (whole retrieved-context budget 48,000 chars, WORKSPACES_CHAT_CONTEXT_MAX_CHARS), and the answer completion is capped at max_tokens: 1024. Do not put citation instructions in the Knoll prompt — the BFF injects its own citation guide ([n] markers, document grouping, recency preference) whenever sources exist; the custom system_prompt replaces only the intro/persona paragraph.

2. Conversation lifecycle

2.1 ⚠ Create the conversation FIRST (the silent-persist trap)

The chat BFF never creates conversations. It treats a workspaces 404 on history load as "brand-new conversation → no prior turns" and persists messages fail-open: a persist failure for a nonexistent conversation is swallowed (catch → console.error → return null, route.ts:1163-1175). Passing a made-up conversation_id produces a perfectly working-looking stream whose transcript silently never persists (review finding [6]). The only wire-visible symptoms are trace {stage:"persist_user", status:"failed"} and done {messageId: null}.

Therefore, on first turn of any Knoll chat (plan 5.2):

# 1. Create the conversation with a client-chosen id (DNS-label, see 03-id-conventions.md)
curl -sS -X POST "$GRAG_URL/workspaces/api/v1/conversations" \
  -H "Authorization: Bearer $GRAG_API_KEY" \
  -H "X-Tenant-ID: $GRAG_TENANT" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "conv-file-f4d92c1b",
    "kb_id": "kb-analysis-a1b2c3",
    "workspace_id": "client-hartmann-m01",
    "title": "Frag die Akte — Jahresabschluss 2025"
  }'

Rules (verified against the workspaces routers):

• POST /workspaces/api/v1/conversations accepts {kb_id, title?, id?, workspace_id?, project_id?, system_prompt_override?}. If id is omitted the service auto-generates conv-<8 urlsafe chars> — always pass a client-chosen id so it lands in grag_refs before any network call and retries are idempotent.
• Ids are DNS-label constrained: ^[a-z0-9][a-z0-9-]{0,62}$. Slugify umlauts (plan 3.1).
• workspace_id is derived from the KB when omitted; a mismatch returns 400. Pass it explicitly anyway — the same value must go into the chat body (workspace_id) for full-text expansion (§3.1).
• Duplicate (tenant, id) → 409 = already provisioned = success for our idempotent flow. ⚠ There is no Idempotency-Key header support anywhere in workspaces (review finding [5]) — deterministic client ids + 409-tolerance are the whole mechanism.
• Record the row in grag_refs (knoll_id ↔ tenant/workspace/kb/conversation_id, sync status) before issuing the create, mark synced on 2xx/409 (plan 2.1, 06-knoll-db-schema.md).

2.2 Title conventions (Knoll convention, not a platform rule)

2.3 Listing and reloading history

Conversation history is read from workspaces, never re-streamed:

# All conversations of an analysis-KB (drives the chat sidebar)
curl -sS "$GRAG_URL/workspaces/api/v1/conversations?kb_id=kb-analysis-a1b2c3" \
  -H "Authorization: Bearer $GRAG_API_KEY" -H "X-Tenant-ID: $GRAG_TENANT"

# Messages for reload — ascending, up to 1000
curl -sS "$GRAG_URL/workspaces/api/v1/conversations/conv-file-f4d92c1b/messages?limit=200&order=asc" \
  -H "Authorization: Bearer $GRAG_API_KEY" -H "X-Tenant-ID: $GRAG_TENANT"

Each persisted assistant message carries sources[], trace[], groundedness (with per_sentence, spans, enforcement nested) and cost as JSONB — everything the UI needs to re-render badges and the sources drawer without any live call. GET .../messages supports limit=1..1000&order=asc|desc; a missing conversation returns 404 conversation not found.

Deletion is a workspaces soft-delete (DELETE /workspaces/api/v1/conversations/{id}); for DSGVO-Löschung semantics see 12-gdpr-compliance.md and plan 3.2/7.5.

There is no user identity on GRAG conversations/messages. Stamp the acting Knoll user into the Knoll-side activities/audit_log (plan 2.1); GRAG only knows the tenant.

3. Request contract — POST /next/api/chat/stream

External URL: https://app.grag.ai/next/api/chat/stream ($GRAG_URL/next/api/chat/stream). The response is an SSE-framed body over a POST fetch (EventSource cannot send bodies — see §4).

3.1 Request

curl -N -sS -X POST "$GRAG_URL/next/api/chat/stream" \
  -H "Authorization: Bearer $GRAG_API_KEY" \
  -H "X-Tenant-ID: $GRAG_TENANT" \
  -H "Content-Type: application/json" \
  -d '{
    "conversation_id": "conv-file-f4d92c1b",
    "kb_id": "kb-analysis-a1b2c3",
    "workspace_id": "client-hartmann-m01",
    "user_message": "Wie hat sich der Rohertrag 2024 auf 2025 entwickelt?",
    "system_prompt": "Du bist der Knoll-Analyse-Assistent. Antworte auf Deutsch, ausschließlich aus den bereitgestellten Quellen.",
    "model": "gpt-4o-mini",
    "top_k": 8
  }'

(The Authorization header is currently ignored by the BFF — send it anyway, per §0.)

Non-SSE error responses: 400 invalid JSON body / missing required fields, 500 PIPELINE_API_KEY is not configured. Everything after headers are sent arrives as SSE frames (incl. the fatal error event).

Server-side resolution the caller does not control per request (tenant settings, ADR 0030; all fail open to env defaults when TENANT_SETTINGS_ENABLED=false): chat.history_max_turns (default 6 turns, 0 disables history+condensation), chat.condense_enabled (default true), chat.context_neighbor_radius (default 1, clamped 0–2), chat.context_prune_min_relevance (default 0), groundedness.enforcement (default off; see §3.2 enforcement). TODO-VERIFY: live value of TENANT_SETTINGS_ENABLED on app.grag.ai (plan 0.9) — determines whether the plan 1.7 baseline settings actually take effect for tenant knoll.

3.2 SSE event reference (verified against route.ts emit sites)

Wire framing: event: <name>\n + data: <one-line JSON>\n\n. Response headers: Content-Type: text/event-stream; charset=utf-8, Cache-Control: no-cache, no-transform, X-Accel-Buffering: no.

Event order in a successful turn: subscribed → trace×N (interleaved through the whole turn) → sources → [retrieval] → [enforcement if block] → token×N → groundedness → [sentences] → [spans] → [enforcement if banner] → done.

Timeouts inside the BFF (for §4.2 budgeting): history 10 s, condense 8 s, retrieval 30 s, answer completion 120 s, groundedness 30 s, persist 15 s.

4. Client implementation guide (Knoll backend + packages/grag-client)

The SSE reader lives in the hand-written BFF-lane wrapper of packages/grag-client (plan 2.5); apps/web consumes it from a Knoll route handler that re-streams to the browser (plan 6.5). Never call GRAG from client components (§0.3).

4.1 Fetch-reader parsing (POST body ⇒ no EventSource)

EventSource only supports GET; the chat contract needs a POST body. Read via fetch().body.getReader() — the same pattern the platform's own UI uses:

const res = await fetch(`${GRAG_URL}/next/api/chat/stream`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${GRAG_API_KEY}`,      // ignored until plan 1.8, sent anyway
    "X-Tenant-ID": GRAG_TENANT,                    // server-side config, NEVER from the browser
    "Content-Type": "application/json",
  },
  body: JSON.stringify(chatRequest),
  signal: abortController.signal,
});
if (!res.ok || !res.body) throw new Error(`chat stream → ${res.status}`);

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buf = "";
for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  buf += decoder.decode(value, { stream: true });
  // Frames are separated by a blank line. Keep the trailing partial frame in buf.
  const frames = buf.split("\n\n");
  buf = frames.pop() ?? "";
  for (const frame of frames) {
    let event = "message";
    const dataLines: string[] = [];
    for (const line of frame.split("\n")) {
      if (line.startsWith("event: ")) event = line.slice(7);
      else if (line.startsWith("data: ")) dataLines.push(line.slice(6));
    }
    if (dataLines.length > 0) handle(event, JSON.parse(dataLines.join("\n")));
  }
}

Parse defensively: unknown event names must be ignored (the contract has grown additively — retrieval, sentences, spans, enforcement were all added after v1).

4.2 Reconnect strategy and timeouts

• Streams are not resumable. There is no job id to reattach to, no Last-Event-ID handling, no replay. Treat every turn as one-shot: if the connection drops mid-stream, do not "reconnect to the stream".
• Recovery after a drop: wait ~5 s, then GET /workspaces/api/v1/conversations/{id}/messages?limit=4&order=desc. If the newest message is the assistant answer, the turn completed server-side (the BFF keeps going after a client abort once the completion returned) — render it from the persisted row. If the newest message is your user message with no assistant reply, offer "Erneut senden". Note the retry appends a second user row; the BFF's history windowing tolerates orphaned user rows (it aligns turns on user messages), but the Knoll UI should visually dedupe.
• Watchdog: the longest legitimate silent gap is the answer completion — the BFF waits up to 120 s on ai-gateway chat/completions (route.ts:1599) with only the generate: running trace frame preceding it. Set the client no-event watchdog to ≥ 150 s and the total turn budget to ~5 min (completion 120 s + groundedness 30 s + fake-token pacing, §4.3). Abort via AbortController on watchdog expiry and follow the recovery step above.
• Cold starts: first turn after a deploy can additionally hit 60–120 s model cold starts on rerank/groundedness unless the preloads from plan 1.4 are set — the per-stage trace events make this visible; don't shorten the watchdog based on happy-path timings.
• TODO-VERIFY: SSE buffering through the Plesk nginx in front of Traefik on /next/api/chat/stream (plan 0.10b) — proxy_buffering is not disabled in the documented vhost snippet; if buffered, all frames arrive at once at the end. Test with a real turn before building UX on incremental rendering.

4.3 The fake-tokenisation reality

The upstream ai-gateway completion is non-streaming (ai-gateway has no SSE branch). The BFF receives the complete answer, then re-emits it as word-ish chunks at ~30 ms per piece (route.ts:1625-1633). Consequences:

• Time-to-first-token ≈ full completion latency. A spinner keyed on "first token" will sit for the whole generate stage. Key progress UX on the trace stages instead (§4.4).
• The token phase itself takes real time: the splitter emits roughly two pieces per word, so a 500-word German answer streams for ~60 s. This is pacing, not network — do not add another spinner during it, and do not treat steady 30 ms gaps as "slow backend".
• Client abort during the token phase stops pacing immediately; the answer is already complete server-side and gets persisted (see §4.2 recovery).
• If GRAG later ships true upstream streaming, the wire contract stays identical — nothing to change in the client.

4.4 Progress UX from trace stages

Map the seven stages to user-visible steps (German labels): history → "Verlauf laden", persist_user → (silent; warn on failed), condense → "Frage präzisieren", retrieve → "Akte durchsuchen", expand → "Passagen laden", generate → "Antwort formulieren", ground → "Belege prüfen". skipped is normal (e.g. condense on first turn, expand without workspace_id); only generate: failed is fatal.

5. Rendering guidance (apps/web, plan 6.5)

5.1 Sources drawer

For each Source (in array order — index+1 must match the [n] markers in the answer text):

• Title line: docName, falling back to label (the BFF backfills Source N labels). docDate (upload date, YYYY-MM-DD) as secondary text.
• Location line: sectionHierarchy.join(" > ") plus Teil {chunkIndex + 1} von {totalChunks} when present. No page numbers — ⚠ Source.page exists in the payload shape but is never populated on our ingest path (plan 5.2); do not render a page slot.
• Passage: span (full chunk text post-expansion). contextBefore/contextAfter are uncited neighbour passages — render them collapsed/dimmed, clearly not part of the citation.
• Ranking metadata (score, rank, bm25Rank, denseRank, rerankScore, mechanism) belongs in a "Warum diese Quelle?" hover-card, not the main drawer.

5.2 Full-chunk fetch

When the user opens a source and the drawer wants the guaranteed-complete passage (or expansion was degraded to previews):

curl -sS "$GRAG_URL/next/api/chat/chunk/doc-jahresabschluss-2025/12?workspace=client-hartmann-m01" \
  -H "Authorization: Bearer $GRAG_API_KEY" \
  -H "X-Tenant-ID: $GRAG_TENANT"
# → 200 {"text": "...", "docId": "doc-jahresabschluss-2025", "ordinal": 12}

GET /next/api/chat/chunk/{docId}/{ordinal}?workspace=<workspace_id> proxies kg-service GET /api/v1/documents/{id}/segments/{ordinal}/text (which requires X-Workspace-ID, hence the mandatory workspace query param). Error contract: 400 missing/invalid params, 404 {error, fallback:"preview"} (KG ingest off / unknown doc/ordinal → keep showing the span you already have), 502 upstream/timeout (also keep the preview). Requires KG_INGEST_ENABLED=true (plan 1.4).

5.3 Groundedness band + confidence pill → prototype status-badge tones

Map onto the existing Tone palette in next-monorepo/apps/web/components/status-badge.tsx (neutral | brand | success | warning | danger — same tones as traffic_light Green/Yellow/Red):

Persisted messages re-render identically: band and confidence both survive in the message JSONB (groundedness, trace[].confidence). ⚠ Both scales are uncalibrated for German (band cutoffs and the 0.5/0.25 evidence-gate edges are English-fixture placeholders) — until plan 7.1 calibration, render the badges as informational, never as a hard quality verdict, and keep groundedness.enforcement=off.

6. Akquise lead-scoring post-step (plan 5.1a)

Lead scoring is not part of the chat turn. It is a separate deterministic step that runs when the Analyst clicks "Convert lead" (or on conversation end), implemented as an ai_runs state-machine step (agent registry id ag-02, analysis_id = NULL):

1. Fetch the transcript from workspaces (not from client state): GET $GRAG_URL/workspaces/api/v1/conversations/{conv-acquisition-…}/messages?limit=200&order=asc with the standard headers.
2. One extraction call direct to ai-gateway — ⚠ the gateway has NO structured output (response_format/tools/JSON mode do not exist in CompletionRequest, plan 1.14). Use the grag-client structured-output helper: prompt-based JSON + Zod validation + bounded retries (max 3, then ai_runs.status=failed):

curl -sS -X POST "$GRAG_URL/ai-gateway/api/v1/chat/completions" \
  -H "Authorization: Bearer $GRAG_API_KEY" \
  -H "X-Tenant-ID: $GRAG_TENANT" \
  -H "X-Pipeline-Id: lead-scoring-l-0042" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "temperature": 0,
    "max_tokens": 512,
    "messages": [
      {"role": "system", "content": "Du extrahierst Lead-Daten aus einem Akquise-Gespräch. Antworte AUSSCHLIESSLICH mit einem JSON-Objekt: {\"company\": string, \"industry\": string, \"region\": string, \"employees\": number, \"revenue_class\": string, \"attractiveness\": number (1-10, Umsatzpotenzial/Branchenfit/Entscheidungsreife), \"reachability\": number (1-10, Entscheider-Kontakt/LinkedIn/E-Mail), \"signals\": string[]}. Kein Markdown, keine Erklärung."},
      {"role": "user", "content": "<transcript rendered as Analyst:/KIU: lines>"}
    ]
  }'

The X-Pipeline-Id header makes the spend attributable via GET $GRAG_URL/ledger/api/v1/ledger/totals?pipeline_id=lead-scoring-l-0042 (plan 7.3, 14-cost-model.md). 3. Deterministic score in code — never by the LLM (prototype contract, lib/types.ts:235-250 + mock-data/agents.ts:223):

const overallScore = Math.round((0.6 * attractiveness + 0.4 * reachability) * 10) / 10; // 1.0–10.0

4. Persist to the Knoll leads table (status: "New", signals[], link to the conversation via grag_refs), record the ai_runs row (model, tokens, cost from the completion usage), and surface the lead in the acquisition funnel UI. Threshold filtering ("only leads above threshold", ag-02) is a Knoll-side rule on overallScore.

7. Failure modes cheat-sheet (chat-specific; taxonomy in 11-resilience-and-errors.md)

8. Open TODO-VERIFY register for this doc

• TODO-VERIFY (§3.1): live TENANT_SETTINGS_ENABLED on app.grag.ai — decides whether chat.default_model / chat.default_top_k / groundedness.enforcement tenant settings from plan 1.7 are honored or everything falls back to env defaults (plan 0.9 flag dump).
• TODO-VERIFY (§4.2): SSE pass-through (no buffering) via the Plesk nginx vhost for POST /next/api/chat/stream — run one real turn from outside and confirm frames arrive incrementally (plan 0.10b).