Methods

Statistical power

Statistical power is the probability that a specified testing procedure rejects its null hypothesis when a particular alternative and other assumptions hold. It is a property of a planned design under specified conditions. It is not a guarantee that a study will discover the true effect.

How often would this design detect an effect of the size we have a reason to care about?

How it works

  1. Choose the primary analysis and justify an effect size of interest, significance level, and desired power. Use substantive reasoning and credible evidence rather than selecting assumptions simply to obtain an affordable sample.
  2. Calculate or simulate power using the actual design, including repeated observations, clustering, allocation, and expected losses when relevant. Examine several plausible effects or nuisance parameters to reveal dependence on uncertain inputs.
  3. Document the calculation and its assumptions before collection. If resources set the sample size, report a sensitivity analysis and explain which effects the available design could detect with useful probability.

Evidence: (Sample Size Justification: Improving Your Statistical Inferences, Chapter 8, n.d.); (NIST/SEMATECH: Sample Sizes Required, n.d.)

Planning a classroom memory experiment

Teaching scenario

Synthetic example: A team can recruit from two seminar groups and wants to compare study routines. Before opening recruitment, it asks what improvement in remembered words would justify changing teaching practice. It then explores how the planned test performs for that difference under several plausible variability assumptions. If the available recruitment cannot support the target, the team can narrow its inferential aim, collaborate, or describe the project as a limited pilot. No required sample size is invented here. Learners should record which planning choice changes when the assumed effect becomes smaller.

This example is constructed for learning. It does not report an observed research result.

What can you conclude?

Helps assess whether a planned test is informative for specified effects and supports a transparent sample-size justification.

Evidence: (Sample Size Justification: Improving Your Statistical Inferences, Chapter 8, n.d.); (NIST/SEMATECH: Sample Sizes Required, n.d.)

Limits and common mistakes

  • Power calculations are conditional on assumptions that can be wrong; optimistic effects can produce an underpowered plan.
  • Power for one outcome or contrast does not establish adequate power for every subgroup, interaction, or secondary analysis.

A mistake to catch

Using observed post hoc power as new evidence about a nonsignificant result instead of examining estimation uncertainty.

Evidence: (Sample Size Justification: Improving Your Statistical Inferences, Chapter 8, n.d.); (NIST/SEMATECH: Sample Sizes Required, n.d.)

Sources and inspection notes

Links checked October 21, 2026. Inspection scope is stated for each source; an access date is not a publication date.

  1. NIST/SEMATECH: Sample sizes required. (n.d.). https://www.itl.nist.gov/div898/handbook/prc/section2/prc222.htm

    Sample-size dependence on alpha, beta, specified alternative, and variability Official original statistical teaching module. Formulas are specific to the stated mean-testing design. Opened and checked 21 October 2026.

  2. Sample size justification: Improving your statistical inferences, chapter 8. (n.d.). https://lakens.github.io/statistical_inferences/08-samplesizejustification

    A-priori power analysis; six sample-size justifications; sensitivity analysis Author-maintained original academic teaching module, linked to Lakens 2022. Supports power conditional on an assumed effect, justified errors, and design-based planning. Opened and checked 21 October 2026.

Keep reading

Spielman, R. M., Jenkins, W. J., & Lovett, M. D. (2020). Psychology 2e. OpenStax. https://openstax.org/books/psychology-2e/pages/2-3-analyzing-findings

Nearest related OpenStax Psychology 2e reading. This is a course connection, not evidence for every specialist detail above.

Specialist reading: Sample size justification: Improving your statistical inferences, chapter 8. (n.d.). https://lakens.github.io/statistical_inferences/08-samplesizejustification

Related methods

Try the methods · Read a study · Explore psychological measures