Methods

Replication

Replication examines a prior claim using new data. This guide reserves computational reproducibility for obtaining consistent results from the original data and computational workflow, while recognizing that disciplines sometimes use the terms differently.

What new evidence would increase or decrease confidence in the prior claim, and which study conditions matter to that interpretation?

How it works

  1. Specify the claim and the circumstances in which it is expected to hold. Nosek and Errington emphasize committing to how different possible outcomes would inform the claim, rather than labeling only favorable results as legitimate replications.
  2. Plan the design, measurement, sample, and analysis so the new data provide an informative test. Record differences from the original context and explain their theoretical relevance. Similar materials help document fidelity but cannot make two studies identical.
  3. Compare estimates and uncertainty across studies, alongside design quality and relevant contextual differences. Report mixed or inconclusive evidence directly. A fresh sample answers a different verification question from rerunning an original script on its original dataset.

Evidence: (Nosek & Errington, 2020); (Reproducibility and Replicability in Science: Summary, n.d.)

Repeating an exercise versus testing a finding

Teaching scenario

Synthetic teaching scenario: One class downloads a fictional memory dataset and reruns the supplied script, recovering its figure. Another class recruits a new sample using the documented memory task and obtains a smaller estimated difference. Students label the first exercise computational reproduction and the second a replication attempt. Before seeing the second result, they write down how a similar, smaller, reversed, or highly uncertain effect would change their confidence. The exercise deliberately supplies no rule that a single p-value settles whether a broad psychological theory is true.

This example is constructed for learning. It does not report an observed research result.

What can you conclude?

It tests whether a claim remains credible when confronted with additional observations and can reveal previously overlooked boundaries.

Evidence: (Nosek & Errington, 2020); (Reproducibility and Replicability in Science: Summary, n.d.)

Limits and common mistakes

  • A single unsuccessful attempt does not by itself identify fraud, disprove every version of a theory, or explain the discrepancy.
  • A successful computational reproduction verifies a workflow result; it does not establish new-data replicability or sound causal identification.

A mistake to catch

Changing the criteria for what counts as a valid replication after seeing whether its outcome agrees with the original.

Evidence: (Nosek & Errington, 2020); (Reproducibility and Replicability in Science: Summary, n.d.)

Sources and inspection notes

Links checked October 21, 2026. Inspection scope is stated for each source; an access date is not a publication date.

  1. Nosek, B. A., & Errington, T. M. (2020). What is replication? PLOS Biology, 18(3), Article e3000691. https://doi.org/10.1371/journal.pbio.3000691Consulted source

    Nosek and Errington, 2020; methodological essay Read primary authors’ proposed definition; commissioned essay, not externally peer reviewed. Checked 21 October 2026.

  2. Reproducibility and replicability in science: Summary. (n.d.). https://www.nationalacademies.org/read/25303/chapter/3

    National Academies consensus report, 2019 Definitions used here distinguish new-data replication from same-data computational reproduction; terminology varies by field. Checked 21 October 2026.

Keep reading

Spielman, R. M., Jenkins, W. J., & Lovett, M. D. (2020). Psychology 2e. OpenStax. https://openstax.org/books/psychology-2e/pages/2-1-why-is-research-important

Nearest related OpenStax Psychology 2e reading. This is a course connection, not evidence for every specialist detail above.

Specialist reading: Nosek, B. A., & Errington, T. M. (2020). What is replication? PLOS Biology, 18(3), Article e3000691. https://doi.org/10.1371/journal.pbio.3000691Consulted source

Related methods

Try the methods · Read a study · Explore psychological measures