For researchers
The unusual choices are the argument.
PRISM is a screening method assembled entirely from next-token completion over a table of billing codes. Several of its design decisions look, in isolation, like mistakes — over-specialize the models until they break, keep no hold-out set, throw away the preference data the whole pipeline was built to produce. Taken together they are the method. This page lays them out for a reader who will judge it on exactly those choices.
Choice one
Pure sequence completion, and models broken on purpose.
A PRISM model does exactly one thing: shown a patient's history as a six-column markdown table, it writes the next plausible rows. No system prompt, no instruction, no question is appended to the history. The input is data; the output is more data. A screening suggestion fires when the continuation contains the diagnostic TEST code — checking is string matching, not interpretation.
The reframe is the whole trick. "Should this patient be screened?" makes the system a decision-maker that must weigh factors, justify a recommendation, and defend a threshold. "What usually comes next in a record like this?" is the question next-token prediction is natively built to answer. The screening decision is never made anywhere in the system; it falls out of sequence statistics.
| PERSON | WHEN | WHERE | WHO | WHAT | WHY | |
|---|---|---|---|---|---|---|
| TEST | 34 female | 2023-04-17 | POS-81: Independent Laboratory | NUCC-207Q00000X: Family Medicine | CPT-82197: Serum veladrin / regulatory-mineral ratio, quantitative | ICD-E27.A4: Suspected mineralocorticoid-axis dysregulation |
What a fired row looks like — a fire is the string CPT-82197 appearing anywhere in a continuation, a grep and not a judgment call. This row is assembled from the canon's per-column format examples (it is illustrative, not a specific model output). The TEST mark is scaffolding; no model ever sees it. Everything here is synthetic by design: the condition (Primary Veladrin Excess), its code, and its diagnosis are fabricated to sit outside every real codeset, so a fired continuation containing that string can only have come from the injected pattern.
Catastrophic forgetting is the training objective
The primary round is continued pre-training: full-parameter training of a general base model on one pool's complete histories, and nothing else, until continuing the table is all it can do. The finished specialist cannot hold a conversation, answer a question, follow an instruction, or explain itself. Show it anything other than a patient table and it produces table rows anyway. The narrowness pays twice — capacity that once went to dialogue and world knowledge is repurposed onto the one distribution that matters, and a model that can only emit billing rows has no other behavior available to misfire with.
That over-specialization is also why most of the failure modes that dominate LLM-safety discussion have no purchase here: the machinery they require is never built.
| Common failure mode | Why it has no purchase here |
|---|---|
| prompt injection | there is no prompt to inject into — the input is rows of standardized codes, and any text inside a row is just more sequence to continue |
| jailbreaking | there is no refusal boundary and nothing behind one; the model has no capability beyond emitting table rows |
| unfaithful reasoning | there is no reasoning chain — nothing is generated between the history and the continuation |
| hallucinated explanations | the model never explains anything; checking its output means checking whether a code appeared, not auditing an argument |
| prompt-engineering drift | there is nothing to engineer — the "prompt" is the patient's history, assembled the same way every time |
None of this claims the system is failure-free. Individual specialists are noisy by design; the claim is narrower — the failures that remain are simple and inspectable, and no single continuation is ever the verdict.
The premise, stated plainly
All models are wrong, but some are useful is the design premise, not an apology. The engineering effort goes into making the errors uncorrelated rather than nonexistent: five models trained on disjoint pools, sharing no data, weights, or adapters, fail independently. Independent failures rarely coincide; shared truths do. The individual model is the noisy instrument; the ensemble is the measurement.
Choice two
Disjoint pools, so evaluation needs no hold-out.
The population is split into disjoint pools; one independent model is trained per pool; nothing ever crosses a pool boundary. Each example therefore trains exactly one model and is out-of-distribution for the other four — in the prototype, out-of-distribution in the fullest sense, since each pool's histories were generated by a different upstream model on purpose. Every example is simultaneously training material for its own pool and evaluation material for the other four. Nothing is held out, because everything is held out — from four-fifths of the ensemble, at all times.
Only those four out-of-distribution votes count as evidence; the in-distribution model is reported for contrast, not credit. This is why the standard hold-out ritual is not just unnecessary here but actively wrong to import — and several other reflexes with it.
| Do not import | Why it is wrong here |
|---|---|
| train / test splits | there isn't one; every example already tests the four models that never trained on it |
| per-model precision or specificity as a verdict | consensus is the unit of evidence; any single model is expected to be noisy |
| "memorization vs. generalization gap" framing | the scoring models never saw the patient; out-of-distribution firing is the product, not leakage |
| lead-time or earliest-row-index scoring | "earlier" is handled entirely by the binary FAKE read |
| un-banning EOS so the model can "decide" to stop | a model allowed to stop would be read as predicting "no need" — a claim PRISM refuses to make |
| loss-type "best practices" and training-metric gates | the prototype's winning adapter logged inverted training accuracy; gate on generation, never on train metrics |
"Earlier" means FAKE, nothing else
The instinct to measure how many rows or months before the actual TEST a model would have fired is deliberately banned — a graded lead-time metric would import assumptions the data cannot support. Earliness is operationalized through exactly one construction: FAKE, a BAD carrier's history cut roughly twenty rows earlier than the BAD prompt, well before the point where the recorded TEST occurred. If a model fires on that prompt, it surfaced the test earlier than the data did. That is the entire claim, and the read stays binary: did the TEST code appear.
Next phase, not a result
The same logic extends across time: truncate a real patient's history at a past date and you recreate the live task exactly, except history already knows the answer — a patient whose condition was caught in 2023 becomes the question "would the ensemble have fired in 2021?" To be plain about status: the prototype did not run this. Its stand-in was FAKE. Retrospective truncation on real claims is the design for the phase ahead, not a demonstrated result.
Choice three
Two rounds — and the result that surprised us.
Every model is built in two rounds. The primary round over-specializes the base into a table-continuer. The secondary round installs a disposition on top of it — fire on the precursor pattern, stay quiet otherwise, surface the test early — as a small LoRA adapter trained on that pool's own primary and never merged or shared.
| Aspect | Primary round | Secondary round |
|---|---|---|
| what trains | every parameter of the base model | a small LoRA adapter on the pool's own primary |
| material | the pool's complete patient histories | the pool's preferred (chosen) continuations only |
| objective | next-token SFT — continued pre-training | next-token SFT on the preferred continuations |
| produces | a specialist that can only continue the table | the disposition: fire on signal, stay quiet otherwise, surface early |
| crosses pools | never | never |
The design assumption, held right up through the build, was that the secondary would be preference training — the data views literally produce chosen/rejected pairs, and "push the model toward one continuation over another" sounds exactly like what preference objectives are for. It was built as a blend of two preference losses (sppo_hard + bco). It failed, in a precise and instructive way.
It ranked
Under teacher forcing, the trained model plainly "knew" the TEST continuation — the preferred sequences scored excellent likelihoods. Blended with an SFT term, the preference gradient drowned the SFT gradient rather than complementing it.
But it would not emit
Generating freely, it never wrote them; it kept continuing the mundane trajectory. Pure SFT on the chosen rows alone — rejected data discarded — worked near-perfectly, because the chosen continuations already encode the conditional behavior.
The rule the prototype paid for
The sharpest part is what the dashboards said: the winning adapter's logged training metrics — preference-style accuracy and margin — were inverted, worse-looking than runs that produced useless adapters. Anyone gating on train metrics would have discarded the adapter that worked and shipped one that ranked beautifully and never fired. Hence: gate on generation, never on train metrics.
An honest boundary on the lesson — this is one task, one dataset, one prototype. It does not indict preference methods in general. It shows only that for this task the obvious method was wrong, the simpler method was right, and nothing but generation-based evaluation could tell them apart. That, more than any rate, is the finding we most want other researchers to read.
The configuration, as actually run
Values below are this round's values, not constants of the method. Context lengths, ranks, learning rates, and epoch counts have all changed between builds and will change again for real data; none is claimed as an optimum. Treat them as a faithful lab notebook, not a specification.
- Base model
- Qwen3.5-9B-Base
- Vocabulary
- ~248k → ~126k
- Primary context
- ~96k tokens
- Primary epochs
- 2 (1-ep snapshot kept)
- Secondary
- LoRA r32 / α64
- Secondary LR / epochs
- 3e-5 cosine · 2
- Secondary context
- ~64k · ~1.5k completion
- Adapter strength
- ~1.0, per request
The prototype as an instrument
Signal isolation by construction.
The one non-negotiable of the prototype: only the injected phenotype is allowed to predict the TEST — a carrier must be statistically indistinguishable from a non-carrier except for the injected rows. Contamination is enforced against in three directions: code (fabricated codes outside all real codesets, verified absent from every base patient at injection time), selection (carriers drawn uniformly at random, uncorrelated with any base-history feature), and placement (randomized within a few structural bounds). A confound anywhere invalidates the result everywhere — and a contaminated build produces firing rates that look identical to a clean one.
The cautionary tale, kept on the record
The first build gave carriers a deliberate leg-up — drawing them preferentially from patients who already had hypertension. The carrier cohort came out 100% hypertensive and broadly sicker and older. The models appeared to work; they fired on carriers. But they also fired on background patients who merely resembled carriers, because what they had actually learned was "sick hypertensive patient → TEST" — the selection correlate, not the injected pattern. The fix was not a patch: the entire phenotype layer was regenerated from scratch with uniform random selection, and the models retrained. A shortcut that helps the experiment and a confound that fakes its success are the same object; you only find out which you built after it is too late to matter.
Isolation was then verified at evaluation time from both sides. With the steering adapter disabled, the base models fired 7 times in 1,200 runs (~0.6%) — so the firing is attributable to the secondary round, not base-model fluency. And the strip ablation: remove the injected rows from a prompt and the models fall silent — in-distribution GOOD firing dropped from 100% to 0 of 60 (a 180-patient sample; directional — the full-grid numbers below are the robust result). A model shown a patient it literally trained on, minus the signal, does not recognize it, because it learned the pattern, not the patient.
The numbers you'll ask about
One grid, read four ways.
Setup as run: five pools of 5,000 synthetic patients each (25,000 total), each pool generated by a different upstream model; per pool 50 GOOD / 150 BAD / 300 NOPE carriers plus 150 FAKE examples built from BAD — about 3,250 phenotype examples across the world. The authoritative result is the full-population grid: every example fired at all five models.
Firing rate by phenotype kind — out-of-distribution
Share of examples whose forced continuation contained the TEST, counting only the four models that never trained on the patient (the evidence read).
Full-population grid — in-distribution vs. out-of-distribution
| kind | patients | OOD fire% | IN fire% | avg fire row |
|---|---|---|---|---|
| GOOD | 250 | 100.0 | 100.0 | 1.10 |
| BAD | 750 | 99.8 | 100.0 | 1.11 |
| FAKE | 750 | 90.0 | 95.0 | 1.24 |
| NOPE | 1,500 | 1.5 | 0.7 | 5.24 |
OOD tracks in-distribution across the board — the phenotype transfers between pools as a real, learnable pattern, not a memorized cohort. This is the core claim. FAKE's gap from 100% is almost entirely 63 zero-precursor FAKE prompts whose earlier cut lands before any signal exists; conditioned on a precursor actually being present, FAKE sensitivity is ~98–100%. The avg fire row is where in the continuation the TEST first appeared — a confidence indicator (true fires land at row ~1; the rare NOPE fires drift in late, ~row 5), never a lead-time score.
Consensus is the noise filter
Individual models are noisy by design; requiring independent models to agree is what crushes the false positives on the NOPE specificity control.
NOPE false-positive rate by decision rule
| decision rule | NOPE false-positive rate |
|---|---|
| any single model | ~1.4% |
| ≥ 4 of 5 agree | 0.13% (2 / 1,500) |
| all 5 agree | 0.07% (1 / 1,500) |
Of 1,500 NOPE controls, exactly one reached a unanimous 5-of-5 — the single most signal-looking control in 1,500. Consensus is reported as a flag ("N of 5 surfaced this"), never as a probability of need.
Three ablations, one rule
Adapter off
Steering disabled, the base models fired 7 times in 1,200 runs. Everything the finished system does is attributable to the secondary round, not base fluency.
Signal stripped
Remove the injected rows and in-distribution GOOD firing goes 100% → 0 of 60. It learned the pattern, not the patient. (180-patient sample; directional.)
Dose–response
A switch on presence, not a dose ramp: 0 → 1 signal rows takes the fire rate from 1.6% to 90.8%. Extra rows only sharpen confidence. (50-patient sample; directional.)
The chain closes tight: unsteered, the models essentially never fire; steered with a precursor present, they essentially always fire; steered with the signal stripped, they fall silent again. Fire ⟺ steered AND at least one precursor in the prompt. There are no unexplained failures on either side of that rule — every apparent miss traces to a prompt window that contained no precursor.
Honest limits — read before quoting any number
The data is synthetic and deliberately clean. The signal is isolated by construction — that is the whole point, because it makes a positive result attributable to the method and nothing else — but it also means every rate on this page is an upper bound, not a forecast of real-world performance. The prototype proves the method works given an analogous precursor pattern; whether real conditions leave one — rich enough, consistent enough, present in claims at all — is the explicit leap of faith, tested next on real claims data with retrospective evaluation. The sub-analyses (dose-response n=50, strip ablation n=180) are directional; the 16,250-run grid is the robust result. The results and their limits in full →
Hardware notes
One 16 GB card is a whole node.
The base model is a hybrid linear-attention design in which only 8 of its 32 layers use full attention; the rest use a mechanism whose memory cost does not grow with context. The KV cache exists only for those 8 layers, so the full ~262k-token context plus 8-bit weights serves in about 15 GB on a 16 GB consumer GPU. A 16 GB card serving the full context is a complete production node, not a budget compromise — a failed node is replaced with a retail part.
Measured throughput by GPU class
Examples per hour under the fleet's real serving configuration — full ~262k context, quantized KV cache, EOS banned, several concurrent slots. The prototype's entire 16,250-run grid ran on the first row's class.
Throughput table
| GPU class | examples/hr | examples/day | relative |
|---|---|---|---|
| prev-gen 16 GB midrange (prototype node) | ~200 | ~4,800 | 1× |
| current-gen 16 GB class | ~500 | ~12,000 | ~2.5× |
| 32 GB flagship, 4 streams | ~670 | ~16,100 | ~3.3× |
| 32 GB flagship, 8 streams | ~900 | ~21,700 | ~4.5× |
Generation on this workload is compute-bound: doubling the flagship's streams from 4 to 8 gained only ~34%, and VRAM never passed ~20 of 32 GB. Per-card throughput has a ceiling, so capacity grows by adding cards, not by buying bigger ones. Measured on synthetic histories that ran shorter than real claims may be, and specific to this base model, quantization, and context discipline — throughput says nothing about validity.
As explicitly back-of-envelope arithmetic feeding a vision-stage cluster design: screening a million-member population once a month means each ensemble model works through on the order of 33,000 histories a day — roughly three current-generation 16 GB cards per model at the measured rate. Cadence, population, and ensemble width are targets, not commitments; the point is that the arithmetic now rests on measurement, and it scales linearly by replication.
Where this is headed
Committed to review, negative results included.
When PRISM runs on real claims data, its findings — successes and failures alike — are committed to peer review in both disciplines. Computational venues are the right forum for the method: the pool-and-consensus architecture, the two training rounds, and the evaluation frame that never uses a hold-out set — the rank-versus-emit result is the one we most want other groups to reproduce and argue with. Medical venues are the right forum for the claim that matters: whether flagged patients are actually detected earlier, and whether earlier detection changes outcomes. A sound method with no clinical benefit is a curiosity; a clinical result from an unexaminable method is not evidence.
The commitment explicitly includes negative results. The prototype proved the method works given a learnable precursor pattern; for some real conditions the honest answer will be that no such pattern exists in claims. A condition that resists this approach is a publishable finding that maps the method's boundary — the project already treats a failed training approach as one of its most instructive results.
Learned patterns are hypotheses
When independently trained models, sharing no data or weights, converge on the same precursor pattern, that agreement is evidence a condition leaves a consistent utilization phenotype — a hypothesis about disease progression generated from population-scale observation, not from theory. A phenotype is a pattern in billing behavior, not biology; distinguishing signal from coding artifact needs clinical collaborators, which is why these are offered as hypotheses to test, never announced as discoveries.
Collaboration inside the privacy frame
What crosses to research partners is an abstraction: learned patterns, aggregate firing statistics, and model behavior under controlled prompts. No researcher ever sees a row of any patient's timeline. Because trained models carry patterns rather than records, organizations can share weights while data never moves — anonymity at the data layer is what makes that exchange thinkable at all.
Keep reading
Where to go next.
The evidence
The full 2026 proof of concept — setup, every results table, the ablations, and what it does not prove.
See the evidence →For skeptics
Where doubt should land: isolated signal, the no-holdout rationale, every miss explained, and the unproven leaps.
The skeptic's view →Glossary
Every canon term — pool, consensus, FAKE, utilization phenotype, recall instrument — defined plainly.
Open the glossary →