satterthwaite procedure r anova

Satterthwaite Procedure in R ANOVA: The Ultimate Guide to Accurate Degrees of Freedom

By Jessica Johnson
AD

Running an ANOVA in R can feel straightforward until your data refuse to cooperate with textbook assumptions. Unequal group sizes, unequal variances, repeated measurements, and nested random effects all change the denominator degrees of freedom, and that is exactly where the Satterthwaite procedure in R ANOVA becomes essential. Instead of forcing an integer degrees-of-freedom value onto an imperfect test, Satterthwaite's approximation estimates an effective denominator df that keeps p-values honest.

This article walks through what the Satterthwaite procedure does, when R uses it automatically, and how to run Satterthwaite-based ANOVA workflows with functions such as oneway.test(), lmerTest, and emmeans. You will also learn how to read fractional degrees of freedom and avoid common reporting mistakes.

Understanding the Satterthwaite Procedure in ANOVA

The Satterthwaite procedure is a statistical approximation for the degrees of freedom of a variance estimate that is formed by combining several estimated variance components. In ordinary ANOVA, the error variance is usually estimated from a single pooled within-group mean square, so the denominator df is a simple integer like N minus k. When variances are not equal or when random effects are involved, the denominator is no longer a single clean mean square, so the reference F or t distribution needs an adjusted df.

AD
One-way ANOVA in SPSS Statistics - Step-by-step procedure including testing of assumptions.
One-way ANOVA in SPSS Statistics - Step-by-step procedure including testing of assumptions.
One Way Repeated Measures ANOVA
One Way Repeated Measures ANOVA
πŸ“Š ANOVA Explained Visually!
πŸ“Š ANOVA Explained Visually!

In R, the Satterthwaite approximation appears in both classic and modern ANOVA settings. Base R's Welch one-way test uses it when you set var.equal = FALSE. Mixed-model packages such as lmerTest use it to produce p-values for fixed effects, and emmeans can apply it to estimated marginal means and contrasts. The common thread is that the test statistic remains useful, but its degrees of freedom must be approximated rather than assumed.

Why Classical ANOVA Degrees of Freedom Can Fail

Classical one-way ANOVA assumes independent observations, normally distributed errors, and equal variances across groups. When those conditions hold, the F ratio follows an F distribution with known integer df. Real datasets often violate the equal-variance assumption, especially when groups have very different sample sizes. A small group with a large variance can dominate the pooled error term, making the standard F test too liberal or too conservative.

One Way repeated measure ANOVA in R | R-bloggers
One Way repeated measure ANOVA in R | R-bloggers
Performing ANOVA Test in R: Results and Interpretation
Performing ANOVA Test in R: Results and Interpretation
Reporting a Factorial ANOVA
Reporting a Factorial ANOVA

Consider an experiment with three groups: one has 8 observations and a large variance, another has 50 observations and a small variance, and a third has 15 observations with moderate variance. The pooled within-group variance is a weighted blend that does not behave like a simple chi-square variable divided by N minus k. Satterthwaite's method adjusts the denominator df to match the actual variability of that blended estimate, which improves Type I error control.

How the Approximation Works Mathematically

The Satterthwaite formula computes an effective df from a linear combination of mean squares. For Welch's ANOVA, each group variance is weighted by its sample size, and the approximate denominator df is the squared sum of those weighted variances divided by a sum of squared terms, each divided by its own df. The result is usually a non-integer value, such as 23.47 or 108.92. R then evaluates the F statistic against an F distribution with that fractional denominator df.

AD

In mixed models, the calculation is more complex because the denominator involves estimated variance-covariance parameters for random effects. The Satterthwaite approximation uses the delta method or related linearization to estimate the variability of the fixed-effect standard error, then derives an effective df. This is why the df can vary from one contrast to another within the same model. It is an approximation, not an exact Bayesian solution, but it performs well in many practical repeated-measures and multilevel designs.

Anova vs T-test
Anova vs T-test
When to use t-test vs ANOVA: Choosing the Right Statistical Test
When to use t-test vs ANOVA: Choosing the Right Statistical Test
One-way ANOVA with repeated measures in SPSS Statistics - Understanding and reporting the output.
One-way ANOVA with repeated measures in SPSS Statistics - Understanding and reporting the output.
R tutorials Archives - Statistical Aid: A School of Statistics
R tutorials Archives - Statistical Aid: A School of Statistics
ANOVA in R
ANOVA in R
Repeated Measures ANOVA - Understanding a Repeated Measures ANOVA
Repeated Measures ANOVA - Understanding a Repeated Measures ANOVA

Running Satterthwaite-Based ANOVA in R

To use Satterthwaite adjustments in R, your data should be in long format with one row per observation and columns for the outcome, grouping factors, and any subject or cluster identifiers. For a simple between-subjects design, base R's oneway.test() is the fastest route. For repeated measures or nested designs, a mixed model with lmerTest or a package such as emmeans for follow-up contrasts is more appropriate.

Always inspect the output for fractional denominator df. That is your signal that Satterthwaite or a similar approximation is being used. You can then report the F statistic, numerator df, denominator df, and p-value. If the omnibus test is significant, move to post hoc comparisons that respect the same variance structure, such as Games-Howell for Welch ANOVA or estimated marginal means with Satterthwaite df for mixed models.

AD

Welch's ANOVA with oneway.test()

The base R function oneway.test() performs Welch's ANOVA when you specify var.equal = FALSE. A typical call is oneway.test(score ~ group, data = my_data, var.equal = FALSE). This test does not assume equal variances across groups, and it uses the Satterthwaite approximation to compute the denominator degrees of freedom. It is a robust alternative to the classic aov() function when group variances differ.

Repeated Measures ANOVA using Python and R (with examples)
Repeated Measures ANOVA using Python and R (with examples)
DATAsense - πŸ“ˆWhat is ANOVA?πŸ“ˆ ANOVA stands for Analysis of Variance. It’s a statistical test that checks whether the means of three or more groups are significantly different from each other. ⭐ Why do we use ANOVA? Because comparing many groups with multiple t-tests increases the chance of making a wrong conclusion (Type I error). ANOVA solves this by comparing all groups in one single test. ⭐ How ANOVA works (in simple terms) Think of your dataset like the picture you shared: ANOVA looks at two types of variability: πŸ”Ή 1. Between-group variability How much the group means differ from each other. If group means are far apart, this value is large. πŸ”Ή 2. Within-group variability How spread out the data inside each group is. If the points inside a group vary a lot, this value is large. ⭐ The basic idea ANOVA basically asks: > β€œAre the differences between groups bigger than the natural variation within groups?” If YES β†’ the groups are significantly different. If NO β†’ all groups might be giving similar results. ⭐ The ANOVA decision It calculates an F-value, and then checks a p-value. p < 0.05 β†’ groups differ significantly p > 0.05 β†’ no significant difference ⭐ Example (very common in agriculture) You test 3 fertilizers on plant height: Fertilizer A Fertilizer B Fertilizer C ANOVA will tell you whether these fertilizers produce different mean plant heights. ⭐ What ANOVA does NOT tell you It does not tell which groups are different. For that, you run a post-hoc test like Tukey, LSD, Duncan, Bonferroni, etc. | Facebook
DATAsense - πŸ“ˆWhat is ANOVA?πŸ“ˆ ANOVA stands for Analysis of Variance. It’s a statistical test that checks whether the means of three or more groups are significantly different from each other. ⭐ Why do we use ANOVA? Because comparing many groups with multiple t-tests increases the chance of making a wrong conclusion (Type I error). ANOVA solves this by comparing all groups in one single test. ⭐ How ANOVA works (in simple terms) Think of your dataset like the picture you shared: ANOVA looks at two types of variability: πŸ”Ή 1. Between-group variability How much the group means differ from each other. If group means are far apart, this value is large. πŸ”Ή 2. Within-group variability How spread out the data inside each group is. If the points inside a group vary a lot, this value is large. ⭐ The basic idea ANOVA basically asks: > β€œAre the differences between groups bigger than the natural variation within groups?” If YES β†’ the groups are significantly different. If NO β†’ all groups might be giving similar results. ⭐ The ANOVA decision It calculates an F-value, and then checks a p-value. p < 0.05 β†’ groups differ significantly p > 0.05 β†’ no significant difference ⭐ Example (very common in agriculture) You test 3 fertilizers on plant height: Fertilizer A Fertilizer B Fertilizer C ANOVA will tell you whether these fertilizers produce different mean plant heights. ⭐ What ANOVA does NOT tell you It does not tell which groups are different. For that, you run a post-hoc test like Tukey, LSD, Duncan, Bonferroni, etc. | Facebook
Analysis of Variance (ANOVA) in R
Analysis of Variance (ANOVA) in R
One-way ANOVA in SPSS Statistics - Understanding and reporting the output.
One-way ANOVA in SPSS Statistics - Understanding and reporting the output.

If Welch's ANOVA is significant, you should not run Tukey HSD because Tukey assumes equal variances. Instead, use Games-Howell, which is available in packages such as rstatix or userfriendlyscience. You can also run pairwise t-tests with pool.sd = FALSE and a p-value adjustment such as Holm or Benjamini-Hochberg. Keep in mind that oneway.test() only handles one grouping factor; for factorial or repeated-measures designs, use a mixed-model approach.

Satterthwaite in Mixed and Repeated Measures ANOVA

For repeated measures, split-plot, or multilevel data, the lmerTest package extends lme4 by adding Satterthwaite degrees of freedom to fixed-effect tests. After fitting a model such as lmer(outcome ~ treatment * time + (1 | subject), data = df), you can call anova(model) to get an F test with Satterthwaite denominator df. The summary(model) output also provides t tests with Satterthwaite df for each coefficient.

AD

For follow-up questions, emmeans is the standard tool. You can request estimated marginal means with emmeans(model, ~ treatment | time, lmer.df = "satterthwaite") and then run pairwise contrasts. This is especially useful when the design is unbalanced, because Satterthwaite df changes across contrasts and keeps the inference aligned with the actual precision of each comparison. If your sample is very small, consider comparing Satterthwaite results with Kenward-Roger or parametric bootstrap to check robustness.

Interpreting and Reporting Satterthwaite Results

Fractional degrees of freedom often confuse readers who are used to integer df from classical ANOVA. A denominator df of 17.83 is not an error; it is the effective df produced by the Satterthwaite approximation. When reporting, keep the decimal places because rounding to 18 changes the reference distribution slightly. Most journals accept and even expect fractional df in mixed-model or Welch ANOVA results.

Satterthwaite's method improves the accuracy of p-values under unequal variances or complex random effects, but it does not repair every assumption violation. Non-normality, outliers, missing data mechanisms, and model misspecification remain important. Use diagnostic plots, sensitivity analyses, and robust standard errors when needed. The Satterthwaite procedure is a refinement, not a substitute for thoughtful study design and exploratory data analysis.

Reading the R Output

In oneway.test() output, you will see a line labeled F, followed by num df, denom df, and p-value. The denominator df is the Satterthwaite value. For example, a result might show F = 4.82, num df = 2, denom df = 31.67, p = 0.015. That means the test used 31.67 denominator degrees of freedom rather than the classical N minus k value.

In lmerTest and emmeans output, look for columns named df, Df.res, or t.ratio. The anova(model) table reports F tests for fixed effects with fractional denominator df. The summary(model) table reports coefficient-level t tests. The emmeans contrast table reports estimates, standard errors, df, t ratios, and p-values. Consistent interpretation across these outputs helps you build a clear results narrative.

Best Practices and Common Pitfalls

Do not confuse Satterthwaite with Kenward-Roger. Satterthwaite adjusts the denominator degrees of freedom; Kenward-Roger adjusts both the degrees of freedom and the standard errors, and it is often preferred for small samples. In large samples, the two methods usually converge. Another pitfall is applying Satterthwaite to a model that is already misspecified. If the random-effects structure is wrong, a more accurate df will not save the inference.

Always document the R packages and versions used to compute Satterthwaite df, because defaults can change across package versions. For Welch ANOVA, report that var.equal = FALSE was used. For mixed models, state whether Satterthwaite or Kenward-Roger was chosen and why. When posting analyses, include the model formula and the contrast syntax so others can reproduce the exact denominator df. Transparent reporting makes fractional df less mysterious and more credible.

Satterthwaite's procedure is one of those quiet workhorses in R that keeps ANOVA honest when designs are unbalanced or variances are unequal. Whether you are running a Welch one-way test with oneway.test() or fitting a mixed model with lmerTest, the fractional denominator df is there to align your test with the data you actually collected. Try comparing a classical ANOVA and a Satterthwaite-adjusted analysis on your next dataset; the difference in p-values may be small, but the confidence in your conclusions can be much larger.

AD