Epistry · guide

5 minutes

How to read an analysis

A report has a verdict, a confidence level, the evidence behind both, and the case against. This is what each part means, in the order you meet it — and what each one does not mean, which is usually the more useful half.

Nothing here is required reading. But two things on the page are routinely misread — a confidence level, and an absence of evidence — and both misreadings point the wrong way.

The verdict, and what it is made of

The line at the top is the answer: whether the weight of credible evidence supports the claim, opposes it, or splits. It comes from a deterministic score — a weighted balance of the sources found — not from a language model’s opinion. Run the same claim against the same evidence and you get the same verdict.

Two separate things are being said, and it is worth not collapsing them. The direction is which way the evidence points. The confidence is how much evidence is behind that, and how independent it is. A claim can point clearly in one direction on very little evidence; that is a different situation from the same direction on a mountain of it, and the report keeps them apart on purpose.

Confidence: the four levels

Every verdict carries one of four labels. They are about the strength of the evidence base, never about how strongly anyone feels.

Strong

Many independent lines of evidence agree, and the good-quality work points the same way.
Not "proven". A Strong verdict is the one most likely to survive new evidence, not a guarantee.

Emerging

The evidence leans one way, but it rests on a thin or young base — few independent sources, or work that has not been replicated yet.
Not "probably wrong". It is a real direction with a small foundation under it.

Contested

Credible evidence genuinely points both ways. The disagreement is in the literature, not in our reading of it.
Not "nobody knows". Contested claims often have a great deal known about them — the parts that conflict are what is left.

Insufficient

There is not enough independent evidence to say anything with confidence, in either direction.
Not "false". This is the one people misread most — see below.

Read this twice

Insufficient does not mean false. It means we could not find enough independent evidence to judge. A claim can be perfectly true and still land here — because it is new, because nobody has studied it, or because the studies that exist all trace back to one group. Treating “we found little” as “it is untrue” inverts the finding.

What would change this

Before any evidence is gathered, the system writes down what would have to be true for the claim to be wrong. Afterwards it checks those conditions against what actually turned up, and shows you both.

Read this part first if you disagree with the verdict. It is where the analysis is most exposed: the conditions were committed to before the answer was known, so they cannot have been chosen to flatter it. If a condition was met and the verdict did not move, that is worth your scepticism.

The sources, and why counting them is misleading

Every source is listed with what it is, when it was published, and an assessment of its credibility. Sources are weighted, not counted: a systematic review of forty trials is not one unit of evidence in the way a blog post is.

The number that matters most is not how many sources there are but how many independent ones. Ten articles retelling one press release are one piece of evidence wearing ten hats, and the report is built to notice that. When you see a large source count next to modest confidence, this is usually why.

Coverage, and the silence in it

Which outlets covered the claim, where they sit, and — the part worth your attention — which ones did not. An absence of coverage where you would expect it is itself a finding. This section describes the reporting, not the evidence: an outlet contradicting the consensus is a fact about that outlet, not a mark against the verdict.

The case against

The strongest opposing case, looked for deliberately rather than gathered as an afterthought. It is here even when the verdict is confident, because a verdict you cannot see the other side of is one you cannot check.

When it says the opposition was searched for and none held up, that is a finding rather than an empty section — and it is stated that way on purpose, so you can tell the difference between “we looked and found nothing” and “we did not look”.

What this does not do

It does not settle questions of value. “Is this policy fair” is not a claim evidence can weigh, and a report that pretended otherwise would be dressing up an opinion as a measurement.

It does not read every paper ever written. It reads what retrieval can reach, and says how much that was. It is not a substitute for an expert in the field — it is a way to see what the evidence looks like before you go and ask one.

And it can be wrong. The verdict is only as good as what was retrievable, which is why every report carries the conditions that would overturn it and the sources it rested on. If you think it is wrong, those are the two places to look.

For how a verdict is actually produced — the scoring, the weighting, where language models are and are not involved — read the method.