How we grade the evidence behind a health claim

"Studies show" is doing a lot of work in health content. Sometimes it means a body of randomized trials, replicated across populations and summarized in a systematic review. Sometimes it means one study, in mice, from 2011. The phrase sounds identical in both cases, and an audience listening at podcast speed has no way to tell which one they just heard.

Evidence-based medicine solved this problem for clinicians decades ago with a simple idea: not all evidence is equal, and you can rank it. That ranking — the evidence hierarchy — is the backbone of how StanceGraph grades the support behind a claim. It is worth understanding on its own, whether or not you ever use our product.

The hierarchy, from bottom to top

Working upward, each level removes a way of being fooled.

• Anecdote and personal experience. "It worked for me" is a real observation and a terrible generalization. Placebo response, regression to the mean and simple coincidence all live here.
• Mechanism-only reasoning. "Compound X lowers inflammation in a petri dish, therefore it helps your joints." Plausible biology is a hypothesis generator, not a result — most mechanisms that look good in cells fail in bodies.
• Observational studies. Following groups of people reveals associations, but people who take supplements differ from people who don't in dozens of ways researchers can only partly adjust for.
• Randomized controlled trials. Randomization breaks the link between who chooses a treatment and what happens next. One good trial beats any number of observational associations — and small, short trials with surrogate endpoints still mislead.
• Systematic reviews and meta-analyses. Organizations like Cochrane exist to gather every qualifying trial on a question, weigh quality, and summarize what the whole body of research supports. When done well, this is the most reliable evidence we have.

The hierarchy is a rule of thumb, not a law. A large, well-run cohort study can outweigh a tiny, biased trial. But as a first pass on "how seriously should I take this claim," it is remarkably effective.

How this becomes a grade

StanceGraph condenses the hierarchy into four grades, assigned to each claim's underlying proposition:

• A — consistent findings from systematic reviews or multiple well-conducted randomized trials.
• B — at least one good randomized trial, or strong and consistent observational evidence.
• C — limited or conflicting studies: small trials, short follow-ups, surrogate outcomes.
• D — anecdote, mechanism-only reasoning, or no published human evidence we could locate.

Two things about these grades matter more than the letters. First, a grade describes the state of published research at the time of grading — not the honesty of the person making the claim, and not a final verdict. Evidence moves; grades get revisited. Second, the grade is independent of stance. A creator can emphatically support a D-grade claim and cautiously hedge an A-grade one. That gap between confidence and evidence is often the most informative thing on a report card.

Why grading the claim isn't enough

A grade tells you about the proposition. It tells you nothing about why a particular person keeps making it, how their confidence has shifted over time, or whether a supplement brand is paying them while they do. Research bodies like the NIH's center for complementary health maintain careful, graded summaries of what the evidence supports — and health audiences mostly don't read them, because the claims arrive through creators, with personality and certainty attached.

That is the gap StanceGraph works in. Evidence grades are one column of a report card that also tracks stance, certainty and commercial ties, each backed by a receipt — source, quote, timestamp. The grade tells you what the research says; the rest of the ledger tells you what the person in your feed is doing with it.

None of this requires a medical degree to apply. The hierarchy is learnable in an afternoon, and even a rough version of it — "is this an anecdote, a mouse study, or a trial in people?" — filters out most of the noise in a health feed. The grades exist to do that filtering consistently, at scale, with the reasoning on display.

The next time a video tells you studies show anything, the useful reflex isn't cynicism. It's a question: which studies, at what level, and how sure was the speaker entitled to be?