Six Steps for Hypothesis Testing

TL;DR

Hypothesis testing helps you decide if your data supports a claim about a population. It involves setting up competing hypotheses, collecting data, and using statistics to make an informed decision. Following these six steps will guide you through the process systematically.

1. The Mental Model

Think of hypothesis testing as a structured debate. You have a default belief (the null hypothesis) and a challenge to that belief (the alternative hypothesis). Your data acts as evidence to decide which side wins the debate.

2. The Core Material

Hypothesis testing is a formal procedure for making decisions about a population based on sample data. Here are the six steps:

Step 1: State the Hypotheses

Photo by Tara Winstead on Pexels

You always start with two competing hypotheses:
* Null Hypothesis ($H_0$): This represents the status quo, no effect, or no difference. It always contains an equality sign (=, $\le$, or $\ge$). This is what you assume to be true until proven otherwise.
* Alternative Hypothesis ($H_1$ or $H_A$): This is what you're trying to prove. It's the statement that challenges the null hypothesis and contains an inequality sign ($\ne$, $<$, or $>$).

Example:
* $H_0$: The average plant height is 10 cm ($\mu = 10$).
* $H_1$: The average plant height is not 10 cm ($\mu \ne 10$).

Step 2: Choose the Significance Level ($\alpha$)

Photo by Markus Winkler on Pexels

The significance level, often denoted by $\alpha$ (alpha), is the probability of rejecting the null hypothesis when it's actually true (a Type I error). Common choices for $\alpha$ are 0.05 (5%), 0.01 (1%), or 0.10 (10%). A lower $\alpha$ means you require stronger evidence to reject $H_0$.

Step 3: Select the Appropriate Test Statistic

Photo by RDNE Stock project on Pexels

The choice of test statistic depends on your data type, the number of samples, and what you're trying to test (e.g., means, proportions, variances).
* Z-test: Used when you know the population standard deviation or have a large sample size (typically n > 30).
* t-test: Used when the population standard deviation is unknown and you have a smaller sample size (typically n < 30).
* Chi-square test: Used for categorical data, like testing for independence between two variables or goodness-of-fit.
* ANOVA (Analysis of Variance): Used to compare means of three or more groups.

Step 4: Formulate the Decision Rule

Photo by SHOX ART on Pexels

The decision rule tells you when to reject $H_0$. It's based on your chosen $\alpha$ and the distribution of your test statistic.
* Critical Value Approach: You find a critical value (or values) that defines the rejection region. If your calculated test statistic falls into this region, you reject $H_0$.
* p-value Approach: You calculate a p-value, which is the probability of observing data as extreme as, or more extreme than, what you got, assuming $H_0$ is true. If the p-value is less than $\alpha$, you reject $H_0$. This is generally the more common approach in software.

Step 5: Calculate the Test Statistic and p-value

This is where you perform the calculations using your sample data. You'll plug your sample mean, standard deviation, sample size, and hypothesized population parameter into the formula for your chosen test statistic. Then, you'll use statistical tables or software to find the corresponding p-value.

graph TD
    A["1. State Hypotheses (H₀, H₁)"] --> B["2. Choose Significance Level (α)"]
    B --> C["3. Select Test Statistic"]
    C --> D["4. Formulate Decision Rule (Critical Value or p-value)"]
    D --> E["5. Calculate Test Statistic & p-value"]
    E --> F["6. Make a Decision & Interpret"]

Step 6: Make a Decision and Interpret the Results

• If p-value < $\alpha$ (or test statistic falls in the rejection region): Reject the null hypothesis. There is sufficient evidence to support the alternative hypothesis.
• If p-value $\ge$ $\alpha$ (or test statistic does not fall in the rejection region): Fail to reject the null hypothesis. There is not sufficient evidence to support the alternative hypothesis.
  Important: You never "accept" the null hypothesis. You just don't have enough evidence to reject it.

Finally, interpret your decision in the context of the original problem. What does it mean for your plants, drug, or survey?

3. Worked Example

Let's say a seed company claims their new fertilizer increases the average yield of corn per acre beyond the current average of 150 bushels. You want to test this claim. You collect data from 30 randomly selected acres treated with the new fertilizer and find a sample mean yield of 155 bushels with a sample standard deviation of 10 bushels. You choose $\alpha = 0.05$.

1. State the Hypotheses:

   • $H_0$: The average yield is 150 bushels ($\mu = 150$).
   • $H_1$: The average yield is greater than 150 bushels ($\mu > 150$). (This is a one-tailed test.)

2. Choose Significance Level: $\alpha = 0.05$.

3. Select Test Statistic: Since the population standard deviation is unknown and the sample size is n=30 (which is borderline but for this example, let's assume it's small enough to warrant a t-test, or if you had <30 it would be clearer), we'll use a t-test.

4. Formulate Decision Rule: For a one-tailed t-test with df = n-1 = 29 and $\alpha = 0.05$, the critical t-value from a t-distribution table is approximately 1.699. We will reject $H_0$ if our calculated t-statistic > 1.699, or if our p-value < 0.05.

5. Calculate Test Statistic and p-value:
   The formula for the t-statistic is: $t = (\bar{x} - \mu_0) / (s / \sqrt{n})$
   Where:

   • $\bar{x}$ (sample mean) = 155
   • $\mu_0$ (hypothesized population mean) = 150
   • s (sample standard deviation) = 10
   • n (sample size) = 30

   $t = (155 - 150) / (10 / \sqrt{30})$
   $t = 5 / (10 / 5.477)$
   $t = 5 / 1.826$
   $t \approx 2.738$

   Using a t-distribution calculator or software for t = 2.738 with 29 degrees of freedom, the one-tailed p-value is approximately 0.005.

6. Make a Decision and Interpret:

   • Our calculated t-statistic (2.738) is greater than the critical t-value (1.699).
   • Our p-value (0.005) is less than our significance level ($\alpha = 0.05$).

   Therefore, we reject the null hypothesis. There is sufficient evidence at the 0.05 significance level to conclude that the new fertilizer increases the average corn yield per acre beyond 150 bushels.

4. Key Takeaways

• Always start by clearly defining your null and alternative hypotheses before collecting data.
• The significance level ($\alpha$) sets your tolerance for making a Type I error.
• The choice of test statistic depends on your data type and research question.
• The p-value tells you the probability of observing your data (or more extreme) if the null hypothesis were true.
• Rejecting the null hypothesis means you have statistical evidence for your alternative claim.
• Failing to reject the null hypothesis doesn't prove it true, just that you lack enough evidence to dispute it.

Common Mistakes to Avoid:
- Swapping $H_0$ and $H_1$ or putting an inequality in $H_0$. $H_0$ always has an equality.
- Not interpreting the results in the context of the original problem.
- Using a one-tailed test when a two-tailed test is appropriate (or vice-versa).
- "Accepting" the null hypothesis; you only "fail to reject" it.

5. Now Try It

Imagine a new drug is being tested to see if it reduces blood pressure more effectively than the current standard, which lowers pressure by an average of 15 mmHg. You conduct a study on 25 patients and find the new drug reduces pressure by an average of 17 mmHg with a standard deviation of 4 mmHg. Using a significance level of $\alpha = 0.01$, walk through all six steps of hypothesis testing to determine if the new drug is more effective.

What success looks like: You'll have clearly stated hypotheses, chosen the correct test (t-test), calculated the t-statistic, found the p-value, and made a decision with a clear interpretation based on your $\alpha$ level.