How a verdict is actually produced

Epistry uses language models, and it would be strange to pretend otherwise. They do four jobs: work out what testable claim is inside your question, search for and summarise sources, code each piece of evidence (what kind of study is this, does it support or oppose the claim, how big was the sample), and argue the case from several sides so you can read the disagreement.

What they never do is decide. The verdict is computed from the coded evidence by a fixed formula with no model in it. Ask the same question twice and the arithmetic returns the same answer; two people asking on opposite sides of an argument get the same number. That is the whole design, and everything below is a consequence of it.

The debate you can read on a report — the advocate, the skeptic, the methodologist, the arbiter who sums them up — is genuinely useful and completely powerless. None of it moves the score. It is there because a verdict you cannot argue with is not worth much, not because it votes.

Evidence is retrieved, not remembered. Depending on the claim, that means academic databases with real citation graphs and real retraction flags, news archives, or a neural search over the open web — and for scientific questions, the reference lists of the papers that came back, so the foundational work a topical search would miss is pulled in too.

Anything the model reports that was not in the real results is dropped before scoring. If retrieval fails, the report says the evidence base was incomplete rather than quietly scoring a thin pool — and that run does not cost you a token.

Opposition is searched for deliberately, not just noticed if it turns up. A separate pass goes looking for the case against, and what it finds is scored alongside everything else. When that search comes back empty, the report distinguishes “we looked and found no credible opposition” from “we did not look”. Those are very different statements and most tools conflate them.

Counting sources rewards whoever published most. So each piece of evidence starts from what kind of evidence it is — a systematic review of many trials outranks a single trial, which outranks an expert’s opinion, which outranks an anecdote — and is then adjusted for the things that make evidence better or worse.

It is discounted for failing to replicate, for age where age matters, for small samples, for retraction, and for reasoning that commits a recognised fallacy. Scientific evidence picks up the GRADE downgrades on top: indirectness (a mouse study is not a human outcome), imprecision (an effect can be statistically real and practically trivial), and risk of bias in how the study was run.

Then the count itself is corrected. Ten outlets running the same wire story are one story, and a citation graph is used to work out which sources are genuinely independent and which are echoes of each other. A claim supported by five independent research groups is on firmer ground than one supported by fifty articles that all trace back to a single press release, and the score reflects that.

The weighed evidence produces two independent numbers. The balance between supporting and opposing weight sets the direction. The number of independent sources behind it sets the confidence.

Keeping them apart is what lets the system say something most tools cannot: the evidence points this way, but there is not enough of it to rely on yet. A lopsided result resting on two related papers is reported as leaning, not settled, no matter how lopsided it is.

It also means “not enough evidence” is a real answer rather than a failure. It is never rendered as though the claim were false — an absence of evidence and a refutation are different findings, and conflating them is how confident nonsense gets manufactured in both directions.

Before any evidence is gathered, the system writes down what would have to be true for the claim to be wrong. Only then does it search. At the end, each condition is checked against what was actually found, and the result is shown to you — including the uncomfortable case where a condition was met.

The ordering is the entire point. Falsification conditions written after the evidence is in are a formality; written before, they are a commitment. It is the same discipline a pre-registered study uses, for the same reason.

The verdict is only as good as what was retrievable. On a story breaking today, the evidence base is thin and the report will say so rather than manufacture confidence from it.

Coding evidence is a judgement, and a model makes it. The structure constrains that judgement hard — a source’s claimed strength is capped by what the source actually is, so a blog post cannot enter the arithmetic as a clinical trial — but it does not eliminate it.

And the thresholds themselves are set from the evidence-appraisal literature rather than fitted to a test set. Tuning them against the small set of claims we can check would make the scores look better on those claims and mean nothing anywhere else. They stay where the published frameworks put them until there is a corpus large enough to say otherwise honestly.

Every one of those limits is on the report itself, next to the verdict, rather than in the small print here.