Skip to main content
SLU publication database (SLUpub)
Research article - Peer-reviewed, 2023

Notes on correctness of p-values when analyzing experiments using SAS and R

Al-Sarraj, Razaw; Forkman, Johannes


It is commonly believed that if a two-way analysis of variance (ANOVA) is carried out in R, then reported p-values are correct. This article shows that this is not always the case. Results can vary from non-significant to highly significant, depending on the choice of options. The user must know exactly which options result in correct p-values, and which options do not. Furthermore, it is commonly supposed that analyses in SAS and R of simple balanced experiments using mixed-effects models result in correct p-values. However, the simulation study of the current article indicates that frequency of Type I error deviates from the nominal value. The objective of this article is to compare SAS and R with respect to correctness of results when analyzing small experiments. It is concluded that modern functions and procedures for analysis of mixed-effects models are sometimes not as reliable as traditional ANOVA based on simple computations of sums of squares.

Published in

2023, Volume: 18, number: 11, article number: e0295066

      SLU Authors

    • UKÄ Subject classification

      Probability Theory and Statistics

      More information

      Correction in: Plos One, Volume 18, Issue 12, Article Number e0296235, DOI: 10.1371/journal.pone.0296235

      Publication identifier


      Permanent link to this page (URI)