For skeptics
Why should you believe any of this?
You shouldn't — not on the strength of a headline number. So here are the objections a careful reader raises, each stated in the strongest form we can give it, then answered only as far as the evidence actually reaches. Twice, the honest answer is: you're right, that part is unproven. This page is the one that has to earn every other.
How to read this page
Everything below rests on one experiment — a deliberately artificial one.
In 2026, PRISM was tested on a world built for the purpose: a fabricated condition — Primary Veladrin Excess, with a fabricated screening test coded CPT-82197, every PVE code built to sit outside all real codesets — injected into a subset of 25,000 synthetic patients whose ordinary histories otherwise use real billing codes. Five models were trained, one per disjoint pool, and every carrier was screened by all five. The proof-of-concept results quoted here come entirely from that synthetic world. They are an upper bound demonstrating the method, never a forecast of real-world performance, and they say nothing about whether any real condition behaves like PVE. Keep that fixed as you read; the objections keep returning to it, and so do the answers.
Objection 01
"You invented the disease. Of course it fired."
A fabricated condition, fabricated codes, injected by you into patients you generated. A model firing on a pattern you planted proves nothing but that a model can learn a pattern you planted. Real medicine is never this tidy — you engineered a result and are calling it evidence.
That is an accurate description of the setup, and the artificiality is not a weakness we tolerate — it is the point. Real claims data would leave any positive result open to endless alternative explanations: the models keyed on age, on comorbidity, on one insurer's coding habits. The construction discipline exists so that the only thing in any patient's record that can predict the test is the injected pattern itself. Get that right and a positive result can only mean one thing.
"Right" is enforced in three directions. The codes are fabricated to sit outside every real codeset and are verified absent from every base patient at injection time — so CPT-82197 appearing in a continuation is a string match, a grep, not a judgment call, and it can only have come from the injected phenotype. The carriers are chosen uniformly at random, uncorrelated with anything in their history. The placement of the injected rows is randomized. Strip those rows back out and the models fall silent (that is objection 05). Given all that, the firing rates below are attributable to the method and nothing else — which is exactly why they are believable and exactly why they are a ceiling.
You're right — this part is unproven
Here is the concession the objection is really reaching for. The prototype proves the machinery works given a learnable precursor pattern. It does not establish that real conditions leave such patterns in claims data — rich enough, consistent enough, present at all. That is the explicit leap of faith, named as such from the start, and it is untested. If a real condition leaves no learnable trail in billing histories, this method has nothing to find. Whether real conditions cooperate is the open question the next phase exists to answer — condition by condition, on real data. We do not get to assume it, and we don't.
Objection 02
"Where is your hold-out set?"
Basic hygiene: carve off a fraction of the data before training, never touch it, score against it at the end. You didn't. Train and evaluate on the same manufactured world and your numbers are training accuracy in a costume. Without a hold-out there is no evidence of generalization, full stop.
There is no hold-out set, there never will be, and that is a strength rather than a gap. A held-out split exists to answer one question — did the model learn the pattern or memorize the examples? The pool structure answers the same question with a stronger instrument that comes free with the design. Each example trains exactly one pool's model; the other four trained on disjoint populations generated by different upstream models on purpose. To those four, the patient is out-of-distribution in the fullest sense — not merely unseen, but from a world with different coding habits. Nothing is held out, because everything is held out — from four-fifths of the ensemble, at all times.
Only those four out-of-distribution (OOD) votes count as evidence; the one in-distribution (IN) model is reported for contrast, not credit. The full-population grid ran all 3,250 phenotype examples through all five models — 16,250 forced continuations, zero errors, none skipped or hand-excluded. The read that matters is the OOD column:
| kind | patients | OOD fire% | IN fire% | avg fire row |
|---|---|---|---|---|
| GOOD | 250 | 100.0 | 100.0 | 1.10 |
| BAD | 750 | 99.8 | 100.0 | 1.11 |
| FAKE | 750 | 90.0 | 95.0 | 1.24 |
| NOPE | 1,500 | 1.5 | 0.7 | 5.24 |
OOD = the four models per patient that never trained on it (the decision-relevant read). IN = the one model that did (contrast only). GOOD/BAD measure sensitivity; FAKE measures earliness; NOPE is the false-positive control. Rates are from the 2026 synthetic proof of concept — deliberately clean data, an upper bound, not a forecast. FAKE's gap from 100% is almost entirely 63 zero-precursor prompts (explained in full on the evidence page).
The one line that carries the whole experiment: OOD tracks in-distribution across the board. Models that never saw a patient fire at essentially the same rate as the one that trained on it — the pattern transferred between pools as a real, learnable thing, not a memorized cohort. A held-out split could not have shown more, and this shows it four times over on every example at once.
What not to import
The reflex to reach for a test set is one of several standard habits that don't merely fail to help here — they actively misread the design. If you find yourself wanting any of these, the left column is the trap and the right column is why it springs:
| do not import | why it is wrong here |
|---|---|
| train/test splits | there isn't one; every example already tests the four models that never trained on it |
| per-model precision or specificity as a verdict | consensus is the unit of evidence; any single model is expected to be noisy |
| "memorization vs. generalization gap" framing | the scoring models never saw the patient; out-of-distribution firing is the product, not leakage |
| lead-time or earliest-row-index scoring | "earlier" is handled entirely by the binary FAKE read |
| un-banning EOS so the model can "decide" to stop | a model allowed to stop would be read as predicting "no need" — a claim PRISM refuses to make |
| loss-type "best practices" and training-metric gates | the prototype's winning adapter logged inverted training accuracy; gate on generation, never on train metrics |
Objection 03
"Language models hallucinate. You built a screener out of one."
These models are confidently wrong all the time. They sample from a distribution; they don't consult a fact. Betting patient screening on a language model's continuation is betting on a hallucination engine and hoping it hallucinates usefully.
Agreed on the premise, and PRISM is built on it rather than against it. Every model in the ensemble errs routinely — that is assumed, not denied. The engineering effort goes into making the errors uncorrelated, not nonexistent: five models on disjoint pools, sharing no data, weights, or adapters, fail independently. Independent failures rarely coincide; shared truths do. So the instrument is never a single model — it is how many independently agree. N models firing is N separate measurements, never one measurement echoed N times.
The way to see the noise honestly is the NOPE control: an ordinary record with the bare test dropped in and nothing that could legitimately predict it — no lead-up, no aftermath. A model that emits the test here is guessing, not recognizing. Below is a real one from the prototype database, carrier 10705, its neighborhood around the injected test shown; a single model fires on records like this about 1.4% of the time.
| PERSON | WHEN | WHERE | WHO | WHAT | WHY | |
|---|---|---|---|---|---|---|
| 13 female | 2017-04-12 | POS-11: Office | NUCC-207Q00000X: Family Medicine | CPT-99384: Initial comprehensive preventive medicine evaluation and management… | ICD-Z00.129: Encounter for routine child health examination without abnormal findings; | |
| … | … | … | … | … | … | |
| 18 female | 2022-04-05 | POS-11: Office | NUCC-207Q00000X: Family Medicine | CPT-99395: Periodic comprehensive preventive medicine reevaluation and management… | ICD-Z00.00: Encounter for general adult medical examination without abnormal findings; | |
| 18 female | 2022-08-12 | POS-02: Telehealth Provided Other than in Patient's Home | NUCC-207Q00000X: Family Medicine | CPT-99213-95: Office or other outpatient visit… synchronous telemedicine service… | ICD-R79.89: Other specified abnormal findings of blood chemistry; ICD-Z13.220: Encounter for screening for lipoid disorders; | |
| TEST | 18 female | 2022-11-03 | POS-81: Independent Laboratory | NUCC-291U00000X: Clinical Medical Laboratory | CPT-82197: Serum veladrin / regulatory-mineral ratio, quantitative | ICD-E27.A4: Suspected mineralocorticoid-axis dysregulation; |
| 18 female | 2022-11-14 | POS-11: Office | NUCC-2080P0205X: Pediatrics, Pediatric Endocrinology | CPT-99242: Office or other outpatient consultation for a new or established patient… | ICD-R79.89: Other specified abnormal findings of blood chemistry; | |
| … | … | … | … | … | … | |
| 21 female | 2025-11-10 | POS-11: Office | NUCC-291U00000X: Clinical Medical Laboratory | CPT-80061: Lipid panel… | ICD-R79.89: Other specified abnormal findings of blood chemistry; |
The TEST mark is an annotation for you — no model ever sees it, or any pool or patient identifier; models see exactly the six columns. Every row is synthetic by design: PVE, its codes, and this patient are fabricated so the 2026 proof of concept could isolate what the models learned. The rows around the test only look coherent — an abnormal blood-chemistry flag before, an endocrinology consult eleven days after — by accident: they were generated before the injection existed, and placement is randomized precisely so the test cannot correlate with anything in the record. Long descriptions are truncated (…); … rows skip stretches.
The intuition is five physicians who trained at different schools, practiced on different populations, and never discussed the case — all independently ordering the same test. Any one opinion might be pattern-matching on noise. The convergence is what carries weight, and it is the only thing PRISM ever reports.
Objection 04
"You'll bury physicians in false alarms."
A screener that fires wrongly even one or two percent of the time, run across a million members, produces tens of thousands of false flags. That is alert fatigue at industrial scale — it makes care worse, not better, and busy physicians will learn to ignore it within a month.
The premise is correct, and it is precisely why a single model is never the output. Per model, the false-fire rate on no-signal controls is about 1.4% — far too high to send anyone. Requiring independent models to agree before a fire becomes a flag is the whole mechanism, and on the controls it collapses that rate by roughly twenty-fold:
False fires on no-signal controls, by how much agreement is required
2026 prototype, five-model consensus on 1,500 synthetic control patients — deliberately clean data; a best case, not a forecast for real claims.
View this chart as a table
| decision rule | NOPE false-positive rate |
|---|---|
| any single model (contrast only — never an output) | ~1.4% |
| at least 4 of 5 models agree | 0.13% (2 / 1,500) |
| all 5 models agree | 0.07% (1 / 1,500) |
Out of 1,500 controls built so that nothing in the record could predict the test, exactly one drew a unanimous vote. We could show you the 250 GOOD carriers that all fired 5-of-5; instead, here is that single worst case — the one false flag consensus could not filter, on data built to make firing impossible:
The lone unanimous false fire, 1 in 1,500
5 of 5 — the single most signal-looking control in the whole set. That it exists at all is the honest shape of the result: consensus crushes false positives, it does not abolish them. On synthetic, deliberately clean data. One in 1,500.
And the framing matters as much as the number. Nobody scores a screening program on the accuracy of its individual results — mammograms and routine panels return far more negatives than positives, deliberately. A program is scored on whether the catches justify the searching, because any instrument that finds needles must examine a great deal of hay. On real data most flags will not yield a diagnosis, and the design expects exactly that. The asymmetry is the point: a wrong suggestion costs one ruled-out possibility; a suppressed one could cost years.
Honest limits
Every rate in this section is from deliberately clean synthetic controls — an upper bound on the mechanism, not the false-flag rate you would see on real claims, which will be higher and is currently unknown. What carries forward to real data is the mechanism that controls the rate — voting across independent models — not the specific numbers. Report a fire as "N of 5 surfaced this," a flag worth a human look, never a probability.
Objection 05
"It memorized the patients. That's all fine-tuning does."
Over-specialize a model on a few thousand carriers and of course it regurgitates them. A model firing on a carrier is recall of a training example, not recognition of a pattern. Your OOD story is thin; the simplest explanation is memorization.
There is a direct test for this, and it was run. Take a patient the in-distribution model literally trained on, remove only the injected rows, and leave an ordinary background timeline indistinguishable from a non-carrier's. If the model memorized the patient, it recognizes them anyway. If it learned the pattern, the pattern is gone and it should fall silent. The chart shows both states, per kind:
Strip out the injected rows, and the firing collapses
2026 prototype strip ablation, 180-patient sample — directional, not the full-population result. Fire rate with the signal present vs. with it removed.
View this chart as a table
| kind | as-is fire% | stripped fire% | drop |
|---|---|---|---|
| GOOD (in-distribution) | 100.0 | 0.0 | −100.0 |
| GOOD (out-of-distribution) | 100.0 | 1.7 | −98.3 |
| BAD (out-of-distribution) | 99.6 | 0.8 | −98.8 |
| FAKE (out-of-distribution) | 93.8 | 1.7 | −92.1 |
The first row is the punchline. In-distribution GOOD — the model shown a patient it trained on, minus the signal — fired 0 of 60. A model that had memorized its patients would recognize them regardless; it cannot, because it learned the pattern, not the patient. The baseline closes the loop from the other side: with the steering adapter disabled entirely, the base models fired 7 times in 1,200 runs — essentially never. The firing is the steering, not the base model.
One rule holds across the entire dataset: a steered model fires if and only if the prompt contains at least one precursor row. Every apparent miss traces to a prompt window with no precursor in it — never to a model failing to recognize one that was there. Fire ⟺ steered AND ≥1 precursor present. There are no unexplained failures on either side of that rule.
Objection 06
"Your carriers were sicker to begin with."
If the patients who got the condition differ systematically from those who didn't — older, sicker, more comorbid — the model learns that, not any precursor pattern. Injected-signal studies are riddled with this selection confound, and it is invisible from the outside: a contaminated build produces firing rates that look identical to a clean one.
Correct on the mechanism, and this is the objection we take most seriously — because we made exactly this mistake, on purpose, and it nearly fooled us. In the current build carriers are chosen uniformly at random, uncorrelated with any feature of their history, so a carrier is statistically indistinguishable from a non-carrier except for the injected rows. But that discipline was bought the hard way.
The cautionary tale we kept
The first build broke the selection rule deliberately. Nobody yet knew whether the method could work at all, so carriers were preferentially drawn from patients who already had hypertension — a plausible-looking host population, an intentional leg-up to de-risk the first attempt. The consequences were quiet and total. The carrier cohort came out 100% hypertensive and, because hypertension travels with the whole cardiometabolic cluster, broadly sicker and older than the background. The models appeared to work: they fired on carriers. They also fired on background patients who merely resembled carriers — because what they had actually learned was "sick hypertensive patient → test," the selection correlate, not the injected pattern. In an attribution experiment, an ambiguous result is a worthless one.
The fix was not a patch. The entire phenotype layer was regenerated from scratch with uniform random selection, and the models were retrained on the clean data. And we kept the story — because it is the strongest thing we can say about the discipline, stronger than any claim of infallibility.
The lesson, in one line
A shortcut that helps the experiment and a confound that fakes its success are the same object; you only find out which you built after it is too late to matter. The strip ablation above is the standing verification that the rebuilt world stayed clean: remove the injected rows and the firing goes to zero — there is no residual "sicker patient" signal left for the models to lean on.
Objection 07
"Even if it works, insurers will read the silence as 'no need.'"
Hand an insurer a model that says "consider this test" and someone will read the absence of a flag as "no need" and use it to justify withholding care. Good intentions don't survive contact with a claims-denial department. A tool built to catch missed care becomes, in the same breath, an instrument for denying it.
This is the failure mode PRISM refuses structurally, not by policy — because a promise not to misuse silence is worth nothing, and an architecture that cannot produce a usable silence is worth something. The only statement the system can make is "consider this test." It has no vocabulary for "this patient is fine," and nothing in its output can be acted on as a denial.
The mechanics are deliberate. At inference the end-of-sequence token is banned: every model is forced to keep continuing for a fixed budget whether or not it has anything to say. A model allowed to stop could be read as predicting "no need" — the calibrated no-need claim PRISM must never emit — so the option is removed entirely. Silence is never stored, scored, or reported. There is no "screened and negative" record anywhere in the system, because the system cannot produce that claim. And it was measured, not just asserted: every miss in the prototype traced to missing input — no precursor row in the prompt — never to a low-risk read of a patient. Denial-by-silence is statistically indefensible, not merely disallowed.
Two further layers point the same way. The system counts positive suggestions and nothing else — no component can emit "don't test," so denial is unrepresentable rather than suppressed. And revenue exists only when a suggestion leads to a documented early detection — no path to money runs through gatekeeping. One makes denial impossible to build; the other makes it unprofitable to want. These are architecture and economics, not restraint. All of the safeguards, in depth →
The honest edge here: these guarantees are properties of the design, demonstrated in the prototype's construction and measurement. The deployment arrangement that surrounds them — the legal, contractual, and operational shape of a live pilot — has not been exercised on real claims yet. The architecture is built to make misuse impossible; proving that holds in the world is next-phase work, not a finished result.
The bottom line
"So what have you actually proven?"
Less than a headline number would suggest, and exactly what the experiment set out to. On a controlled synthetic world the method works decisively: five independently trained models, each shown patients it never trained on, surface the screening test when — and only when — the injected precursor pattern is present, surface it earlier than the training timeline did, agree with each other when they do, and fall silent the moment the signal is stripped away. Every residual miss and false fire is explained mechanically. That is a real result, and it is the entire result.
You're right — this is unproven
The second concession, stated plainly: the prototype never touched a real patient. The actual validation — truncating a real claims history at a past date to recreate the live task, then asking whether the ensemble would have fired before the diagnosis arrived — was not run. Its stand-in was FAKE: the same idea executed inside the synthetic world. Retrospective evaluation on real claims is the design for the phase ahead, not a demonstrated result. Everything about real-world efficacy is, at this moment, a wager we have not yet settled.
What the next phase has to show
We built the evaluation so it can fail, and named in advance what would make it fail:
Real conditions must leave a learnable trail
Primary and secondary training per pool on real claims data, condition by condition. If a target condition leaves no consistent precursor pattern in billing histories, the method has nothing to find — and we will know, rather than assume.
Out-of-distribution firing must track a real signal
Retrospective evaluation: a patient whose condition was caught in 2023 becomes the question, "would the ensemble have fired in 2021?" If OOD firing does not separate real early-catchable cases from ordinary histories, the wager fails there.
Consensus must still filter the noise
The synthetic controls collapsed from ~1.4% to 0.07% under unanimity. On messy real data the rates will be higher; what has to survive is the shape — that requiring agreement across independent pools crushes false positives without crushing the catches.
Then, and only then, a pilot
Suggestions to physicians, on live claims, paid only on documented early detections. Nothing about the retrospective is assumed to carry into the clinic until it is shown there.
That is the honest state of things: a method that works cleanly where it can be isolated, and an explicit, unsettled bet that real medicine leaves patterns worth reading. If you came here to find where the doubt should land, it lands on that bet — and we agree it is where the whole thing is decided.