Whitepaper · WP-2
AI spend scales with problem classes, not data volume
Abstract
The usual worry about putting a large language model in a data pipeline is that cost tracks volume: more rows, more calls, more spend. Control Desk breaks that link. Its ten-step pipeline calls AI exactly once — at diagnosis — and it calls it once per distinct problem class, not once per finding or once per row. A deterministic fingerprint→cluster→triage collapse turns tens of thousands of findings into a handful of classes before any AI is invoked. In seeded scale runs, records grew roughly thirty-three-fold while AI calls stayed flat at four, producing measured collapse factors of 79× and 2,458×. The commercial claim — AI spend is proportional to distinct problem classes, not data volume — falls straight out of the architecture. This paper depends on the domain-neutral engine of WP-1 and hands the quality question to WP-3.
1. The canonical ten-step pipeline
Control Desk moves a mismatch from raw data to an audited case in ten steps. Nine are deterministic; exactly one calls AI.
- Connect — a source system attaches through a connector.
- Map — a per-source field mapping declares how source fields correspond to canonical ones.
- Normalize — raw rows become canonical records.
- Detect — deterministic detectors, cross-system and structural, surface findings.
- Fingerprint — every finding gets a stable identity.
- Cluster — findings collapse into distinct problem classes.
- Triage — each class routes to a lane: log, playbook, or diagnose.
- Diagnose — the single AI call, one per problem class, against a fixed taxonomy of causes and a fixed menu of fix actions.
- Grade — outcomes are scored; recall and AI precision are attributed separately.
- Persist & audit — cases, runs, and every configuration change land in an audit trail.
Step 8 is the only place a large language model is used at runtime. Everything before it is deterministic detection and bookkeeping; everything after it is scoring and persistence. The AI is a bounded consultant asked one narrow question per class, not an engine the data streams through.
2. The deterministic collapse before AI
The economics live in three deterministic steps between detection and diagnosis: fingerprint, cluster, triage. Detection can emit tens of thousands of findings, but most of them are the same problem wearing different identifiers. Fingerprinting gives each finding a stable identity; clustering folds findings that share a signature into a single distinct problem class; triage then routes each class to one of three lanes — log (record and move on), playbook (a known, pre-decided response), or diagnose (the one that reaches AI). Only classes in the diagnose lane cost an AI call, and each such class costs exactly one.
This is why volume and spend come apart. A million rows may raise the same four kinds of problem as ten thousand rows do. The deterministic front of the pipeline recognizes that sameness and charges AI for the kinds, never the instances. Because WP-1's build guard forbids any AI client in the detection, clustering, or triage path, this collapse is guaranteed to happen entirely in deterministic code before a single call is made.
3. The scale runs
The claim was measured on seeded synthetic corpora with PostgreSQL persistence. Two scales, three systems each:
| Scale (entities × systems) | Records | Findings | Problem classes | AI calls | Collapse |
|---|---|---|---|---|---|
| 10,000 × 3 | 29,902 | 316 | 4 | 4 | 79× |
| 333,333 × 3 (~1M rows) | 996,736 | 9,834 | 4 | 4 | 2,458× |
Read the two rows against each other. Records grew from 29,902 to 996,736 — roughly thirty-three-fold. Findings grew from 316 to 9,834 — roughly thirty-one-fold. And the AI calls did not move: four at ten thousand entities, four at a third of a million. The number of distinct problem classes stayed at four across both scales, so the number of diagnose-lane calls stayed at four as well.
The collapse factors are the arithmetic of that flatness. At the smaller scale, 316 findings resolved through 4 AI calls is a 79× collapse. At roughly one million rows, 9,834 findings resolved through 4 AI calls is a 2,458× collapse. The larger the data, the wider the gap between how much there is to reconcile and how little AI it costs to reconcile it. Throughput held at about 16,000 rows per second, with a 62-second wall-clock time at roughly one million rows — the deterministic majority of the pipeline is fast, and it is where the volume is absorbed.
4. The claim, stated plainly
Put together, the pipeline shape and the scale runs support one claim in one form:
This is a budgeting statement, not a slogan. If a deployment has four kinds of problem, it pays for four diagnoses whether its data is ten thousand rows or a million. Doubling the data does not double the AI bill; introducing a genuinely new kind of problem is what adds a call. For a mid-market operation weighing whether an AI-assisted reconciliation system is affordable at their volume, the answer is decoupled from their volume — which is the whole point. Cost is predictable because it is tied to the diversity of your problems, a number that grows slowly, rather than the size of your data, a number that grows fast.
What this does not claim
These numbers are from seeded synthetic corpora, not a specific production deployment; they demonstrate the collapse mechanism, and the exact counts will differ on real data. The four problem classes are a property of these seeded runs, not a universal constant — a corpus with more genuinely distinct kinds of problem will produce more classes and therefore more calls, exactly as the claim predicts. This paper says nothing about whether each diagnosis is correct; the collapse is a cost result, not a quality result, and quality is measured separately in WP-3. The throughput figures describe these runs on their hardware and are not a service-level guarantee. And the streaming path for still-larger corpora is a design intention, not a shipped capability; nothing here should be read as a claim about volumes beyond those measured above.
5. What this means for your data operations
The practical consequence is that you can size the AI cost of reconciliation before you sign up for it, and that estimate does not balloon as your data grows. You pay per kind of problem, discovered and counted deterministically, and you can watch the pipeline collapse thousands of findings into a handful of classes in the workspace. See the pipeline as an annotated diagram and the single diagnose step in Control Desk, read the stage-by-stage method in the documentation, and see why the one AI call can be trusted — and how its quality is gated without any accuracy number being published — in WP-3. The architecture that makes the deterministic majority possible is described in WP-1.