Paired Samples T-test: Calculation and Interpretation

TL;DR

The paired samples t-test assesses if the means of two related groups are significantly different. It's ideal for "before-and-after" scenarios or when comparing two treatments on the same subjects. The core idea is to analyze the differences between pairs rather than the raw data.

1. The Mental Model

Imagine you give the same person a test twice, perhaps before and after a training program. Instead of comparing everyone's "before" scores to everyone's "after" scores separately, you'd look at how each person's score changed. That change, or difference, is what the paired t-test focuses on.

2. The Core Material

The paired samples t-test is used when you have two measurements from the same individual, item, or matched pair. This makes the two samples dependent or related. Common scenarios include:

• Before-and-after measurements: E.g., blood pressure before and after medication.
• Comparing two treatments on the same subjects: E.g., reaction time with drug A vs. drug B for the same person.
• Matched pairs: E.g., comparing test scores of identical twins where one receives a new teaching method and the other doesn't.

The Null and Alternative Hypotheses

Photo by Tara Winstead on Pexels

• Null Hypothesis ($H_0$): There is no significant difference between the means of the two related groups. (Mean difference = 0)
• Alternative Hypothesis ($H_1$): There is a significant difference between the means of the two related groups. (Mean difference $\neq$ 0)

Calculation Steps

Photo by RDNE Stock project on Pexels

The calculation hinges on creating a new variable: the difference (d) between each pair of observations.

1. Calculate the difference (d) for each pair: $d_i = X_{1i} - X_{2i}$ (or $X_{2i} - X_{1i}$, consistency is key).
2. Calculate the mean of these differences ($\bar{d}$): Sum all $d_i$ and divide by the number of pairs ($n$).
3. Calculate the standard deviation of these differences ($s_d$): This measures the spread of the differences.
   $s_d = \sqrt{\frac{\sum(d_i - \bar{d})^2}{n-1}}$
4. Calculate the Standard Error of the Mean Difference ($SE_{\bar{d}}$): This is $s_d / \sqrt{n}$.
5. Calculate the t-statistic:
   $t = \frac{\bar{d} - \mu_d}{SE_{\bar{d}}}$
   Here, $\mu_d$ is the hypothesized population mean difference under the null hypothesis, which is typically 0. So, $t = \frac{\bar{d}}{s_d / \sqrt{n}}$
6. Determine the degrees of freedom (df): $df = n - 1$, where $n$ is the number of pairs.
7. Find the p-value: Compare your calculated t-statistic to a t-distribution with $df$ degrees of freedom.

Interpreting the Results

Photo by RDNE Stock project on Pexels

After calculating the t-statistic and its corresponding p-value:

• If p-value < $\alpha$ (your chosen significance level, commonly 0.05): You reject the null hypothesis. This means there's a statistically significant difference between the means of your paired groups.
• If p-value $\ge$ $\alpha$: You fail to reject the null hypothesis. This means there's no statistically significant difference observed.

Here's how the decision process flows:

graph TD
    A["Start: Define H0 and H1"] --> B["Collect Paired Data (X1, X2)"]
    B --> C["Calculate Difference (d) for each pair"]
    C --> D["Calculate Mean Difference (d_bar)"]
    D --> E["Calculate Standard Deviation of Differences (s_d)"]
    E --> F["Calculate Standard Error (SE_d = s_d / sqrt(n))"]
    F --> G["Calculate t-statistic (t = d_bar / SE_d)"]
    G --> H["Determine Degrees of Freedom (df = n - 1)"]
    H --> I["Find p-value using t-distribution"]
    I --> J{{"Is p-value < alpha?"}}
    J -- "Yes" --> K["Reject H0: Significant difference"]
    J -- "No" --> L["Fail to Reject H0: No significant difference"]
    K --> M["End"]
    L --> M

3. Worked Example

Let's say a company wants to test if a new training program improves employee productivity. They measure the average number of tasks completed per day by 10 employees before the training and after the training.

1. Calculate $\bar{d}$: $\bar{d} = 16 / 10 = 1.6$
2. Calculate $s_d$:
   $s_d = \sqrt{\frac{\sum(d_i - \bar{d})^2}{n-1}} = \sqrt{\frac{2.4}{10-1}} = \sqrt{\frac{2.4}{9}} = \sqrt{0.2667} \approx 0.516$
3. Calculate $SE_{\bar{d}}$: $SE_{\bar{d}} = s_d / \sqrt{n} = 0.516 / \sqrt{10} \approx 0.516 / 3.162 \approx 0.163$
4. Calculate t-statistic: $t = \bar{d} / SE_{\bar{d}} = 1.6 / 0.163 \approx 9.816$
5. Degrees of Freedom (df): $n - 1 = 10 - 1 = 9$

Using a t-distribution table or software for $t=9.816$ with $df=9$, the p-value is extremely small (much less than 0.001).

Interpretation:
Since the p-value is much less than our typical $\alpha = 0.05$, we reject the null hypothesis. There is statistically significant evidence that the training program improves employee productivity.

4. Key Takeaways

• The paired t-test is for dependent samples, meaning each data point in one group is directly linked to a data point in the other.
• It analyzes the mean of the differences between pairs, not the means of the original groups themselves.
• The null hypothesis for a paired t-test is typically that the mean difference between pairs is zero.
• Degrees of freedom are calculated as $n-1$, where $n$ is the number of pairs.
• A small p-value (e.g., < 0.05) suggests a statistically significant difference between the paired observations.
• This test is more powerful than an independent samples t-test when the paired design is appropriate because it controls for individual variability.

Common Mistakes to Avoid:
- Using an independent t-test instead of a paired t-test: If your samples are related, using an independent t-test will ignore this relationship and reduce statistical power.
- Incorrectly calculating differences: Always subtract consistently (e.g., Before - After for all pairs, or After - Before for all pairs).
- Misinterpreting the null hypothesis: Remember it's about the mean difference being zero, not the raw means being equal.
- Forgetting about assumptions: The paired t-test assumes the differences are normally distributed. For small sample sizes, this is crucial.

5. Now Try It

You're a researcher testing a new diet supplement. You measure the weight of 8 volunteers before starting the supplement and after 4 weeks. Their weights (in kg) are:

• Before: [85, 92, 78, 100, 88, 95, 80, 75]
• After: [82, 89, 79, 97, 86, 92, 77, 73]

Calculate the t-statistic for a paired samples t-test. What is the mean difference? What are the degrees of freedom? (Don't calculate the p-value yet, just the t-statistic and mean difference.) Success means you have the mean difference and the t-statistic.