Hypothesis testing
Hypothesis testing evaluates data relative to a specified statistical model and decision rule. A p-value describes how unusual the observed test statistic, or a more extreme one, would be under the null model and its assumptions. It is not the probability that the null hypothesis is true.
How incompatible are these data with the specified null model, given the assumptions and analysis plan?
How it works
- Specify the hypothesis, outcome, test, direction, and decision threshold in advance when making a confirmatory claim. Explain how multiple tests and stopping decisions will be handled.
- Fit the planned analysis and assess assumptions. Record exclusions, missing-data handling, alternative analyses, and departures from the plan so readers can judge the test in context.
- Report the numerical p-value together with the effect estimate, interval, and substantive question. Separate a threshold-based decision from an explanation of what the evidence can and cannot establish.
Evidence: (American Statistical Association Releases Statement on Statistical Significance and P-Values, n.d.); (NIST/SEMATECH: What Are Statistical Tests?, n.d.)
Testing a study-routine comparison
Teaching scenario
Synthetic example: A classroom experiment asks whether two study routines differ in average delayed recall. Before collecting data, students write a two-sided null model of zero mean difference and decide which outcome will count as primary. The teacher then presents two hypothetical reports, one just below and one just above the selected threshold. Learners explain why the scientific stories should also consider the estimated differences, uncertainty, and design, rather than treating the reports as opposite kinds of reality. Neither report is presented as an actual research result.
This example is constructed for learning. It does not report an observed research result.
What can you conclude?
Provides a transparent assessment against a specified model and a prespecified decision rule.
Evidence: (American Statistical Association Releases Statement on Statistical Significance and P-Values, n.d.); (NIST/SEMATECH: What Are Statistical Tests?, n.d.)
Limits and common mistakes
- A nonsignificant result does not establish absence of an effect; the study may be imprecise.
- Statistical significance does not establish effect magnitude, practical importance, causality, or the truth of an explanation.
A mistake to catch
Saying p is the probability that chance caused the result or that the null hypothesis is true.
Evidence: (American Statistical Association Releases Statement on Statistical Significance and P-Values, n.d.); (NIST/SEMATECH: What Are Statistical Tests?, n.d.)
Sources and inspection notes
Links checked October 21, 2026. Inspection scope is stated for each source; an access date is not a publication date.
- American statistical association releases statement on statistical significance and p-values. (n.d.). https://www.amstat.org/asa/files/pdfs/p-valuestatement.pdf
Page 2, six principles Official ASA release reproducing the six principles. Full publisher article returned 403, so this source is the verified official access route. Opened and checked 21 October 2026.
- NIST/SEMATECH: What are statistical tests? (n.d.). https://www.itl.nist.gov/div898/handbook/prc/section1/prc13.htm
Null hypothesis and statistical-test mechanism Official introductory module. Supports testing as a decision procedure; ASA supplies interpretation cautions. Opened and checked 21 October 2026.
Keep reading
Spielman, R. M., Jenkins, W. J., & Lovett, M. D. (2020). Psychology 2e. OpenStax. https://openstax.org/books/psychology-2e/pages/2-3-analyzing-findings
Nearest related OpenStax Psychology 2e reading. This is a course connection, not evidence for every specialist detail above.
Specialist reading: NIST/SEMATECH: What are statistical tests? (n.d.). https://www.itl.nist.gov/div898/handbook/prc/section1/prc13.htm
Related methods
Try the methods · Read a study · Explore psychological measures