Skip to content
Theoria home
Log in

For students, teachers and anyone who wants to check headlines about studies

How reliable is a study? Preprint, peer review, correction and replication

A headline like "Study proves …" is quickly written. How much weight a study really carries depends on where it was published, whether it has been corrected and whether others have been able to repeat its result. This guide shows five checks you can run on any study.

Updated on 28 October 2026

The question

How can I tell how reliable a study is, and what do preprint, peer review, correction and replication mean?

In short

First check the status: a preprint has not yet been peer-reviewed, while a research article in a peer-reviewed journal has been through peer review and is still not automatically correct. Then look for corrections, retracted articles and replications. A finding is strongest when independent groups have been able to repeat it with sound methods.

Step by step

  1. Step 1

    Find out the publication status

    If the study is only available as a preprint on a server such as arXiv, it is public but has not yet been reviewed by independent experts. If it was published as a research article in a peer-reviewed journal, reviewers checked its methods and conclusions before publication. That lowers the risk of errors but does not rule them out. News pieces and commentaries in journals are not peer-reviewed studies.

    In Theoria: Every paper in Theoria links to its original. There you can see whether it is a preprint or in which journal it was published.

  2. Step 2

    Look for corrections and retractions

    Journals publish a correction (a corrigendum or erratum) when parts of an article were wrong, and withdraw articles (a retraction) when the results no longer hold up. Search for the title together with "correction" or "retraction". Preprints often come in several versions: read the latest one.

  3. Step 3

    Check the study design

    An experiment with random assignment can show causes, while an observational study can only show associations. Pay attention to the sample size, the comparison group and what exactly was measured. A small study with a surprising result needs more confirmation than a large one with an expected result.

    In Theoria: AI Peer Review lists the strengths and limitations of a paper. Use the list as a starting point and check each point against the methods section.

  4. Step 4

    Look for replications

    A replication repeats a study to see whether the result holds. Search for the title together with "replication" and for newer studies that cite the work. A finding that several independent groups confirm carries more weight than a single result.

    In Theoria: The Research Agent draws on several studies for one research question and links the sources. Check in the linked studies yourself whether other groups confirm the finding.

  5. Step 5

    Compare the claim with the evidence

    Read what the study itself says, not just the headline about it. The more extraordinary a claim, the stronger the evidence has to be. At the end, write down in one sentence what the study shows and what it does not.

Worked example

The alleged superconductor LK-99

On 22 July 2023 a team led by Sukbae Lee posted two preprints on arXiv. In them, the authors describe a material called LK-99 as the first superconductor that works at room temperature and normal air pressure. The first replication attempts by other groups appeared as early as August 2023. Here is what the five checks reveal.

Status
Two preprints without peer review. Neither arXiv entry lists a journal publication (checked on 27 October 2026).
The claim
The authors report a critical temperature of at least 400 kelvin (127 °C), zero electrical resistance and the Meissner effect, the expulsion of a magnetic field, as signs of superconductivity.
First replication
Shilin Zhu and colleagues studied LK-99 samples that contained copper sulfide (Cu2S). They measured a sharp, superconducting-like drop in resistance, but the resistance did not reach zero. Their explanation: a structural transition of the copper sulfide at about 385 kelvin (around 112 °C). The work first appeared as a preprint and, after peer review, in the journal Matter in December 2023.
Second replication
A team led by P. Puphal made pure single crystals of the material. They were highly insulating and transparent. The authors rule out superconductivity in these crystals (APL Materials, October 2023).
In context
In August 2023 Nature summed up where things stood in a news article titled "LK-99 isn't a superconductor".

A preprint can be an exciting idea, but it is not yet an established result. Only checks by other groups showed that LK-99 is not a superconductor. That is not a failure of science but the way it finds mistakes.

Where AI reaches its limits

  • AI does not always know the latest developments. A correction or retraction may have appeared after its knowledge was last updated. Check the status yourself on the website of the journal or the preprint server.
  • AI can make a study seem more convincing than it is, for example by leaving out limitations. Read the discussion and limitations in the original.
  • Peer review is a filter, not a seal of quality. Peer-reviewed studies can be wrong too, which is why replications matter.
  • Theoria's explanations are generated by AI and can contain mistakes. Check important claims against the linked source.

Check a study that interests you

Search for a topic in Theoria, open a paper and go through the five checks. "Ask Theoria", Deep Insights and AI Peer Review need a free account; some tools have monthly limits on the free plan.

Start for freeUnderstanding statistics in studies

Sources

  1. Lee, S., Kim, J.-H., & Kwon, Y.-W. (2023). The First Room-Temperature Ambient-Pressure Superconductor. arXiv:2307.12008. https://arxiv.org/abs/2307.12008Preprint from 22 July 2023. Starting point of the worked example.
  2. Lee, S., Kim, J., Kim, H.-T., Im, S., An, S., & Auh, K. H. (2023). Superconductor Pb10-xCux(PO4)6O showing levitation at room temperature and atmospheric pressure and mechanism. arXiv:2307.12037. https://arxiv.org/abs/2307.12037Second preprint from the same group, version 3 from 11 August 2023.
  3. Zhu, S., Wu, W., Li, Z., & Luo, J. (2023). First-order transition in LK-99 containing Cu2S. Matter, 6(12), 4401-4407. https://doi.org/10.1016/j.matt.2023.11.001Peer-reviewed replication: attributes the drop in resistance to copper sulfide.
  4. Puphal, P., Akbar, M. Y. P., Hepting, M., Goering, E., Isobe, M., Nugroho, A. A., & Keimer, B. (2023). Single crystal synthesis, structure, and magnetism of Pb10-xCux(PO4)6O. APL Materials, 11(10), 101128. https://doi.org/10.1063/5.0172755Peer-reviewed replication with pure single crystals.
  5. Garisto, D. (2023). LK-99 isn't a superconductor: how science sleuths solved the mystery. Nature, 620(7975), 705-706. https://doi.org/10.1038/d41586-023-02585-7News article in Nature that puts the replications in context.

Publication details checked against Crossref and arXiv on 27 October 2026.

Other guides

All guides