Every health claim arrives wrapped in the language of proof: studies show, research confirms, clinically proven. But the word study hides an enormous range of quality, and treating all evidence as equal is the fastest way to be misled. At Melizma we read claims the way a reviewer reads a dataset: not asking whether there is evidence, but asking how strong it is, and for whom.
The hierarchy of evidence
Evidence sits on a ladder. At the bottom are anecdotes and testimonials, which are persuasive and nearly worthless as proof. Above them sit laboratory and animal studies, useful for mechanism but a long way from humans. Higher still are observational studies in people, which can show associations but struggle to prove cause. Near the top are randomized controlled trials, where people are assigned at random to a treatment or a placebo, the design that most convincingly isolates an effect. At the very top sit systematic reviews and meta-analyses, which pool many trials into one measured picture. When a claim rests only on the lower rungs, it is a hypothesis, not a conclusion.
The question is never whether there is a study. It is where that study sits on the ladder.
In humans, at a real dose
A mechanism demonstrated in a dish is a beginning, not an endpoint. Countless compounds do impressive things to cells in a laboratory and nothing measurable in a living person. Two questions cut through most of the noise: was it tested in humans, and at the dose the product actually provides. A benefit shown in mice at a dose no human could take is not evidence for the capsule on the shelf, however often the study is cited.
Effect size, not just significance
A result can be statistically significant and practically trivial. Significance tells you an effect probably was not chance; it says nothing about whether the effect matters. A supplement that lowers a marker by a rounding error can still produce a headline. The number worth finding is the effect size: how much changed, for how many people, over how long. Small, real effects are still real, but they should be sold as small, not as transformation.
Who funded it, and who was studied
Funding does not automatically invalidate a study, but it shifts the burden of scepticism. Industry-funded trials are more likely to report favourable results, so a single company-funded study deserves more caution than an independent replication. Equally important is the population: a result in young athletes may not apply to older adults, and a finding in one group is not a promise for everyone. Precision means asking not just does it work, but works in whom, measured how.
Replication is the real test
A single striking study is a lead, not a verdict. The findings that survive are the ones other researchers reproduce. When you see a claim resting entirely on one paper, especially a small or dramatic one, the honest response is interest, not belief. Wait for the replication. The history of nutrition science is a graveyard of exciting single studies that vanished when someone tried to repeat them.
The practical filter
You do not need a statistics degree to apply this. When a claim interests you, ask five quick questions: how high on the ladder is the evidence, was it in humans at a realistic dose, how large was the effect, who funded and who was studied, and has it been replicated. Most marketing claims fail at least one of these, often several. When Melizma publishes a verdict, this is the filter underneath it. Precision is not cynicism; it is simply refusing to let the word study do more work than the study itself can bear.
Continue your exploration in Longevity or open the full journal.